Using Glance/Nova on multinode compute

I also tried setting my S3_URL to http://10.239.220.10:9292

This is what I got for client side:

root@nova-cc01:~# uec-publish-tarball ttylinux-uec-amd64-12.1_2.6.35-22_1.tar.gz test x86_64
Wed Apr 20 16:20:19 EDT 2011: ====== extracting image ======
kernel : ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz
ramdisk: ttylinux-uec-amd64-12.1_2.6.35-22_1-initrd
image : ttylinux-uec-amd64-12.1_2.6.35-22_1.img
Wed Apr 20 16:20:20 EDT 2011: ====== bundle/upload kernel ======
Traceback (most recent call last):
  File "/usr/bin/euca-upload-bundle", line 231, in <module>
    main()
  File "/usr/bin/euca-upload-bundle", line 214, in main
    bucket_instance = ensure_bucket(conn, bucket, canned_acl)
  File "/usr/bin/euca-upload-bundle", line 92, in ensure_bucket
    bucket_instance = create_bucket(connection, bucket, canned_acl)
  File "/usr/bin/euca-upload-bundle", line 81, in create_bucket
    return connection.create_bucket(bucket, policy=canned_acl)
  File "/usr/lib/pymodules/python2.6/boto/s3/connection.py", line 321, in create_bucket
    raise S3ResponseError(response.status, response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 404 Not Found
404 Not Found

The resource could not be found.

failed to upload bundle to test/ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz.manifest.xml
failed: euca-upload-bundle --bucket test --manifest /tmp/uec-publish-image.f1ovSs/ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz.manifest.xml
Checking bucket: test
Warning: failed to parse error message from AWS: <unknown>:1:0: syntax error
Creating bucket: test
Warning: failed to parse error message from AWS: <unknown>:1:0: syntax errorfailed to upload kernel

This the glance side:

2011-04-20 16:21:03 DEBUG [routes.middleware] Matched GET /images/detail
2011-04-20 16:21:03 DEBUG [routes.middleware] Route path: '/images/detail', defaults: {'action': u'detail', 'controller': <glance.server.Controller object at 0x1d41ad0>}
2011-04-20 16:21:03 DEBUG [routes.middleware] Match dict: {'action': u'detail', 'controller': <glance.server.Controller object at 0x1d41ad0>}
2011-04-20 16:21:03 DEBUG [eventlet.wsgi.server] 10.239.220.10 - - [20/Apr/2011 16:21:03] "GET /images/detail HTTP/1.1" 200 540 0.007298
2011-04-20 16:21:03 DEBUG [routes.middleware] Matched GET /images/detail
2011-04-20 16:21:03 DEBUG [routes.middleware] Route path: '/images/detail', defaults: {'action': u'detail', 'controller': <glance.server.Controller object at 0x1d41ad0>}
2011-04-20 16:21:03 DEBUG [routes.middleware] Match dict: {'action': u'detail', 'controller': <glance.server.Controller object at 0x1d41ad0>}
2011-04-20 16:21:03 DEBUG [eventlet.wsgi.server] 10.239.220.10 - - [20/Apr/2011 16:21:03] "GET /images/detail HTTP/1.1" 200 540 0.006362
2011-04-20 16:21:04 DEBUG [routes.middleware] Matched GET /images/detail
2011-04-20 16:21:04 DEBUG [routes.middleware] Route path: '/images/detail', defaults: {'action': u'detail', 'controller': <glance.server.Controller object at 0x1d41ad0>}
2011-04-20 16:21:04 DEBUG [routes.middleware] Match dict: {'action': u'detail', 'controller': <glance.server.Controller object at 0x1d41ad0>}
2011-04-20 16:21:04 DEBUG [eventlet.wsgi.server] 10.239.220.10 - - [20/Apr/2011 16:21:04] "GET /images/detail HTTP/1.1" 200 540 0.007064
2011-04-20 16:21:04 DEBUG [routes.middleware] No route matched for GET /test/
2011-04-20 16:21:04 DEBUG [eventlet.wsgi.server] 10.239.220.10 - - [20/Apr/2011 16:21:04] "GET /test/?max-keys=0 HTTP/1.1" 404 176 0.000544
2011-04-20 16:21:04 DEBUG [routes.middleware] No route matched for PUT /test/
2011-04-20 16:21:04 DEBUG [eventlet.wsgi.server] 10.239.220.10 - - [20/Apr/2011 16:21:04] "PUT /test/ HTTP/1.1" 404 176 0.000453

I have the same problem.
I can add image through glance, but i can't use euca-upload-bundle, uec-publish-tarball to upload. Below is my error message.

"uec-publish-tarball" error message:
--------------------------------------------------------
root@clouder:~#uec-publish-tarball /data/images/ubuntu1010-UEC-localuser-image.tar.gz ubuntu1010 amd64 Mon Apr 25 18:02:44 CST 2011: ====== extracting image ======
Warning: no ramdisk found, assuming '--ramdisk none'
kernel : maverick-server-uec-amd64-vmlinuz-virtual
ramdisk: none
image : maverick-server-uec-amd64.img
kernel : maverick-server-uec-amd64-vmlinuz-virtual
ramdisk: none
image : maverick-server-uec-amd64.img
rock 1 read: 200: arg count
Mon Apr 25 18:03:17 CST 2011: ====== bundle/upload kernel ======
failed to register maverick-server-uec-amd64-vmlinuz-virtual.manifest.xml
failed: euca-register --name maverick-server-uec-amd64-vmlinuz-virtual ubuntu1010/maverick-server-uec-amd64-vmlinuz-virtual.manifest.xml
UnknownError: An unknown error has occurred. Please try your request again.failed to upload kernel

"euca-upload-bundle" error message:
-------------------------------------------------------------------------------
euca-upload-bundle -b images -m /tmp/maverick-server-uec-amd64.img.manifest.xml
Checking bucket: images
Uploading manifest file
Warning: failed to parse error message from AWS: http://www.w3.org/TR/html4/strict.dtd:81:4: error in processing external entity reference
Permission denied while writing: maverick-server-uec-amd64.img.manifest.xm

Hi Dennis,

I'm trying to do the same thing right now. As I understand it, if you want to use uec-publish-tarball or the euca-bundle-* commands then you need to run nova-objectstore. Those commands require an S3 API and nova-objectstore is the service that provides it.

However, when I run nova-objectstore and do an upload the image doesn't get stored in Swift. Instead the parts get stored in the usual image directory /var/lib/nova/buckets/ dir.

My relevant nova.conf config is:

--s3_host=127.0.0.1
--glance_host=127.0.0.1
--image_service=nova.image.glance.GlanceImageService

(I'm keeping everything on one machine for the moment while I'm testing, hence the 127 address)

Of course I haven't even been able to store my images in Swift using glance-add (see https://answers.launchpad.net/glance/+question/154253) so I might have something else misconfigured.

Would you be willing to share your glance config so I can compare it with my config? If I can get glance-add working, I'll be able to move on to getting the uec-publish-tarball and euca-bundle-* commands working with Glance/Swift and can try to continue to help.

Everett

Try using glace-upload to upload an image. There are a few image style formats out there (Raw/tar/AMI) and it can get confusing. You will need to identify the type of image you have in order to upload use it correctly.

This is what I did to get the ttylinux-uec-amd64-12.1_2.6.35-22_1.tar.gz to work.

Untar it first. Then upload it as an AMI image:

glance-upload --type=ramdisk ttylinux-uec-amd64-12.1_2.6.35-22_1-initrd ttyinitrd
glance-upload --type=kernel ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz ttykernel
glance-upload --type=machine --kernel=14 --ramdisk=13 ttylinux-uec-amd64-12.1_2.6.35-22_1.img ttyimage

This seemed to work well for me.

It is my understanding nova-object should not run if you are running with glance. It should all go through glance. I just do not think the API is yet compatible with euca-bundle/uec yet. I'm running swift object store to store all my images and nova-object is local machine only storage.

What authentication type are you using in swift? I added the auth-server services and using that instead swauth. If your using swauth I think the URL is different in the glance.conf

ie

https://<PROXY_HOSTNAME>:11000/1.0

vs. (swauth)

https://<PROXY_HOSTNAME>:8080/auth/v1.0

> It is my understanding nova-object should not run if you are running
> with glance. It should all go through glance. I just do not think the
> API is yet compatible with euca-bundle/uec yet. I'm running swift object
> store to store all my images and nova-object is local machine only
> storage.

I want to clarify something here. Nova-objectstore exists only to support euca-upload-bundle and to make euca-register work properly. It works fine with glance image service and local image service.

Nova-objectstore is just a a very simple bucket service (replicates s3 api).

There seems to be quite a bit of confusion on this point, so let me describe how the AWS image registration works by describing what each of the euca commands does.

euca-bundle-vol/euca-bundle-image:
 * compresses the volume/image
 * encrypts the compressed file
 * breaks the encrypted file into 10 mb chunks
 * creates a manifest.xml file with metadata about the image and all of the chunks

euca-upload-bundle:
 * uploads manifest and xml to an s3 bucket

euca-register
 * grabs the manifest from an s3 bucket
 * downloads all of the part files
 * combines the part files into one file
 * decrypts the file
 * decompresses the file
 * extracts metadata from the manifest
 * makes the image available (this involves uploading the decompressed file into the image service along with image metadata)

As you can see, the AWS model for uploading images requires an S3 service to make uploading work. Therefore, to run these commands, you need to have some sort of S3 service. Nova-objectstore is just an ultra-simple s3 service to support this particular set of commands.

Vish

>
> --
> You received this question notification because you are a member of
> Glance Core, which is an answer contact for Glance.

I understand a bit better now. Thank you. So in order for euca-bundle/etc to work both glance and nova-object store need to be running because those tools need a S3 service.

Does the image upload get placed into glance's backend? When I tried running both glance and nova-object, the location of the image never referenced a swift URL (glance details).

 So images what were uploaded using glance-add/upload were in swift while images using euca toolset were referenced local. Is there a configuration setting for nova-object that I missed?

if you specify:
--glance_host=<ip of glance>
--image_service=nova.image.glance.GlanceImageService
in your flagfile, the euca commands, will end up storing the final decrypted image and metadata in whatever backend you have glance configured to use.

Vish

On Apr 26, 2011, at 11:50 AM, Dennis DeMarco wrote:

> Question #153638 on Glance changed:
> https://answers.launchpad.net/glance/+question/153638
>
> Status: Answered => Open
>
> Dennis DeMarco is still having a problem:
> I understand a bit better now. Thank you. So in order for euca-
> bundle/etc to work both glance and nova-object store need to be running
> because those tools need a S3 service.
>
> Does the image upload get placed into glance's backend? When I tried
> running both glance and nova-object, the location of the image never
> referenced a swift URL (glance details).
>
> So images what were uploaded using glance-add/upload were in swift
> while images using euca toolset were referenced local. Is there a
> configuration setting for nova-object that I missed?
>
> --
> You received this question notification because you are a member of
> Glance Core, which is an answer contact for Glance.

one typo in my description above. this line:

* uploads manifest and xml to an s3 bucket

should say:

* uploads manifest.xml and part files to an s3 bucket