problem verifying glance

Asked by Don Mize

I've been going round and round with this. Been all through the questions and tried on two different machines and ended up
at the same spot below. Any ideas?

root@xnode9:~/images# glance --os_username=adminUser --os_password=nova --os_tenant_name=openstackDemo --os_region_name=RegionOne --os_auth_url=http://172.17.86.9:5000/v2.0 add name="tty-linux-kernel" disk_format=aki container_format=aki < ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz
Uploading image 'tty-linux-kernel'
Failed to add image. Got error:
The request returned 500 Internal Server Error

The response body:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 336, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 203, in __call__
    return app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 299, in __call__
    response = request.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1086, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1055, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 299, in __call__
    response = request.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1086, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1055, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 299, in __call__
    response = request.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1086, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1055, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 299, in __call__
    response = request.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1086, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1055, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 318, in __call__
    for line in req.body_file:
TypeError: 'LimitedLengthFile' object is not iterable

Note: Your image metadata may still be in the registry, but the image's status will likely be 'killed'.
========================================================================================================================[100%] 41.7M/s, ETA 0h 0m 0s

Question information

Language:
English Edit question
Status:
Solved
For:
Glance Edit question
Assignee:
No assignee Edit question
Solved by:
Don Mize
Solved:
Last query:
Last reply:
Revision history for this message
smallma (s-rain) said :
#1

Can you paste the following information?

1. keystone and glance version
2. glance-registry-paste.ini, [paste_deploy] in glance-registry.conf glance-api-paste.ini and [paste_deploy] in glance-api.conf
3. export | grep OS in client node
4. endpoint-list in keystone

Regards,

Rain.

Revision history for this message
Don Mize (donald-mize) said :
#2

keystone 2012.1-0ubuntu1

glance 2012.1-0ubuntu2

glance-registry-paste.ini:
------------------------------------

# Default minimal pipeline
[pipeline:glance-registry]
pipeline = context registryapp

# Use the following pipeline for keystone auth
# i.e. in glance-registry.conf:
# [paste_deploy]
# flavor = keystone
#
[pipeline:glance-registry-keystone]
pipeline = authtoken context registryapp

[app:registryapp]
paste.app_factory = glance.common.wsgi:app_factory
glance.app_factory = glance.registry.api.v1:API

[filter:context]
context_class = glance.registry.context.RequestContext
paste.filter_factory = glance.common.wsgi:filter_factory
glance.filter_factory = glance.common.context:ContextMiddleware

[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
service_protocol = http
service_host = 127.0.0.1
service_port = 5000
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
auth_uri = http://127.0.0.1:5000/
admin_tenant_name = service
admin_user = glance
admin_password = glance
---------------------------------------------------

Glance-registry.conf:
---------------------
[paste_deploy]
flavor = keystone
---------------------

Glance-api.conf:
----------------
[paste_deploy]
flavor = keystone
-----------------

There was no paste deploy in glance-api-paste.ini, the install and deploy document didn't say anything about it.

root@xnode9:/etc/glance# export | grep OS
declare -x LESSCLOSE="/usr/bin/lesspipe %s %s"

root@xnode9:/etc/glance# keystone --token ADMIN --endpoint http://172.17.86.9:35357/v2.0 endpoint-list
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+
| id | region | publicurl | internalurl | adminurl |
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+
| 5f780a73764f4c828b57b91366cb75b9 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| 653a2bf1b3b04985b9d140b9626f8797 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| 78186b626031484f8dab4de08753a7b5 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| a1c8ef113861443a8a5bac01ff44bb5e | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+

Thanks!
Don

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of smallma
Sent: Tuesday, April 17, 2012 4:11 AM
To: Mize, Donald
Subject: Re: [Question #193860]: problem verifying glance

Your question #193860 on Glance changed:
https://answers.launchpad.net/glance/+question/193860

    Status: Open => Needs information

smallma requested more information:
Can you paste the following information?

1. keystone and glance version
2. glance-registry-paste.ini, [paste_deploy] in glance-registry.conf glance-api-paste.ini and [paste_deploy] in glance-api.conf
3. export | grep OS in client node
4. endpoint-list in keystone

Regards,

Rain.

--
To answer this request for more information, you can either reply to
this email or enter your reply at the following page:
https://answers.launchpad.net/glance/+question/193860

You received this question notification because you asked the question.

Revision history for this message
Don Mize (donald-mize) said :
#3

I've discovered this in the glance-registry.log

2012-04-17 14:12:23 3978 ERROR [glance.registry.db.api] (ProgrammingError) (1146, "Table 'glance.images' doesn't exist") 'SELECT images.created_at AS images_created_at, images.updated_at AS images_updated_at, images.deleted_at AS images_deleted_at, images.deleted AS images_deleted, images.id AS images_id, images.name AS images_name, images.disk_format AS images_disk_format, images.container_format AS images_container_format, images.size AS images_size, images.status AS images_status, images.is_public AS images_is_public, images.location AS images_location, images.checksum AS images_checksum, images.min_disk AS images_min_disk, images.min_ram AS images_min_ram, images.owner AS images_owner, images.protected AS images_protected \nFROM images \n LIMIT %s' (1,)
2012-04-17 14:12:23 3978 ERROR [glance.registry.db.api] Could not ensure database connection and consistency. Ensure database configuration and permissions are correct and database has been migrated since last upgrade by running 'glance-manage db_sync'

I use the glance-manage db_sync command and I still get the error. The database does have something in it, although I don't see the glance.images table. I'm missing a configuration step I think.

-----Original Message-----
From: Mize, Donald
Sent: Tuesday, April 17, 2012 11:35 AM
To: '<email address hidden>'
Subject: RE: [Question #193860]: problem verifying glance

keystone 2012.1-0ubuntu1

glance 2012.1-0ubuntu2

glance-registry-paste.ini:
------------------------------------

# Default minimal pipeline
[pipeline:glance-registry]
pipeline = context registryapp

# Use the following pipeline for keystone auth
# i.e. in glance-registry.conf:
# [paste_deploy]
# flavor = keystone
#
[pipeline:glance-registry-keystone]
pipeline = authtoken context registryapp

[app:registryapp]
paste.app_factory = glance.common.wsgi:app_factory
glance.app_factory = glance.registry.api.v1:API

[filter:context]
context_class = glance.registry.context.RequestContext
paste.filter_factory = glance.common.wsgi:filter_factory
glance.filter_factory = glance.common.context:ContextMiddleware

[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
service_protocol = http
service_host = 127.0.0.1
service_port = 5000
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
auth_uri = http://127.0.0.1:5000/
admin_tenant_name = service
admin_user = glance
admin_password = glance
---------------------------------------------------

Glance-registry.conf:
---------------------
[paste_deploy]
flavor = keystone
---------------------

Glance-api.conf:
----------------
[paste_deploy]
flavor = keystone
-----------------

There was no paste deploy in glance-api-paste.ini, the install and deploy document didn't say anything about it.

root@xnode9:/etc/glance# export | grep OS
declare -x LESSCLOSE="/usr/bin/lesspipe %s %s"

root@xnode9:/etc/glance# keystone --token ADMIN --endpoint http://172.17.86.9:35357/v2.0 endpoint-list
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+
| id | region | publicurl | internalurl | adminurl |
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+
| 5f780a73764f4c828b57b91366cb75b9 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| 653a2bf1b3b04985b9d140b9626f8797 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| 78186b626031484f8dab4de08753a7b5 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| a1c8ef113861443a8a5bac01ff44bb5e | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+

Thanks!
Don

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of smallma
Sent: Tuesday, April 17, 2012 4:11 AM
To: Mize, Donald
Subject: Re: [Question #193860]: problem verifying glance

Your question #193860 on Glance changed:
https://answers.launchpad.net/glance/+question/193860

    Status: Open => Needs information

smallma requested more information:
Can you paste the following information?

1. keystone and glance version
2. glance-registry-paste.ini, [paste_deploy] in glance-registry.conf glance-api-paste.ini and [paste_deploy] in glance-api.conf
3. export | grep OS in client node
4. endpoint-list in keystone

Regards,

Rain.

--
To answer this request for more information, you can either reply to
this email or enter your reply at the following page:
https://answers.launchpad.net/glance/+question/193860

You received this question notification because you asked the question.

Revision history for this message
smallma (s-rain) said :
#4

Hello Don,
First, you should stop glance and remove DB. And then execute "glance-manage db_sync". at last, you set up glance-api and glance-registry again.

BTW, you endpoint looks like incorrect. Please refer to the following keystone command. and I don't know why there was no paste deploy in glance-api-paste.ini, it's stange. you should check installation steps again. glance-api-paste.ini is very important.

keystone endpoint-create --region RegionOne --service_id XXXXXXXXXXX --publicurl http://172.17.86.9:9292/v1 --adminurl http://172.17.86.9:9292/v1 --internalurl http://172.17.86.9:9292/v1

Regards,

Rain

Revision history for this message
Don Mize (donald-mize) said :
#5

Changing the endpoint fixed the problem. However, I used glance in default DB mode, which means using sqlite. I did try dropping the DB and doing the glance-manage db_sync. It did put markers in the DB for glance, but none of the tables were there.

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of smallma
Sent: Tuesday, April 17, 2012 10:01 PM
To: Mize, Donald
Subject: Re: [Question #193860]: problem verifying glance

Your question #193860 on Glance changed:
https://answers.launchpad.net/glance/+question/193860

smallma posted a new comment:
Hello Don,
First, you should stop glance and remove DB. And then execute "glance-manage db_sync". at last, you set up glance-api and glance-registry again.

BTW, you endpoint looks like incorrect. Please refer to the following keystone command. and I don't know why there was no paste deploy in glance-api-paste.ini, it's stange. you should check installation steps again. glance-api-paste.ini is very important.

keystone endpoint-create --region RegionOne --service_id XXXXXXXXXXX --publicurl http://172.17.86.9:9292/v1 --adminurl http://172.17.86.9:9292/v1 --internalurl http://172.17.86.9:9292/v1

Regards,

Rain

--
You received this question notification because you asked the question.

Revision history for this message
Don Mize (donald-mize) said :
#6

The endpoints were wrong.