Keystone with SSL does not seem to work on Grizzly

Asked by Alfred Shen

Tried to enable SSL for Keystone on Grizzly. Here is the configuration.

On /etc/keystone/keystone.conf
...
[ssl]
enable = True
certfile = /etc/keystone/ssl/certs/signing_cert.pem
keyfile = /etc/keystone/ssl/private/signing_key.pem
ca_certs = /etc/keystone/ssl/certs/ca.pem
#key_size = 1024
#valid_days = 3650
#ca_password = None
cert_required = False
#cert_subject = /C=US/ST=Unset/L=Unset/O=Unset/CN=www.example.com
....

Verified all *.pem files are in place and correct. Restarted keystone-all and ports 5000
 and 35357 are up.

The following ENVS have been defined on the client side.

root@control:/etc/keystone# env | grep OS
OS_PASSWORD=password
OS_CERT=/etc/keystone/ssl/certs/signing_cert.pem
OS_AUTH_URL=http://127.0.0.1:5000/v2.0
OS_USERNAME=admin
OS_TENANT_NAME=demo
OS_KEY=/etc/keystone/ssl/private/signing_key.pem
OS_CACERT=/etc/keystone/ssl/certs/ca.pem

Tried to run keystone client but it hanged... no error was thrown

root@control:/etc/keystone# keystone --debug user-list
REQ: curl -i http://127.0.0.1:5000/v2.0/tokens -X POST -H "Content-Type: application/json" -H "User-Agent: python-keystoneclient"
REQ BODY: {"auth": {"tenantName": "demo", "passwordCredentials": {"username": "admin", "password": "password"}}}
...
<hang>..
...

Tried to run curl but it hanged as well.

root@control:/etc/keystone# curl --cert /etc/keystone/ssl/certs/signing_cert.pem --cacert /etc/keystone/ssl/certs/ca.pem http://127.0.0.1:5000/v2.0/tokens -X POST -H "Content-Type: application/json" -H "User-Agent: python-keystoneclient" {"auth": {"tenantName": "demo", "passwordCredentials": {"username": "admin", "password": "password"}}}

<hang>....

Your assistance is greatly appreciated.

Question information

Language:
English Edit question
Status:
Answered
For:
OpenStack Identity (keystone) Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Koji (kj-tanaka) said :
#1

Hi,

What do you get if you try --os-auth-url https://127.0.0.1:5000/v2.0/tokens instead of http://127.0.0.1:5000/v2.0/tokens ? You would probably need to update the OS_AUTH_URL on your rc file.

Bests,

Revision history for this message
Alfred Shen (alfredcs) said :
#2

After changed to https as suggested it displayed "Authorization Failed". Please see following messages. Meanwhile openssl displayed correct server cert.

root@control:~# keystone --debug user-list
REQ: curl -i https://127.0.0.1:5000/v2.0/tokens -X POST -H "Content-Type: application/json" -H "User-Agent: python-keystoneclient"
REQ BODY: {"auth": {"tenantName": "demo", "passwordCredentials": {"username": "admin", "password": "password"}}}

(eventlet.wsgi.server): 2013-04-23 12:23:40,543 DEBUG wsgi write (32415) accepted ('127.0.0.1', 45733)

Authorization Failed: <attribute 'message' of 'exceptions.BaseException' objects> (HTTP Unable to establish connection to https://127.0.0.1:5000/v2.0/tokens)
root@control:~#
root@control:~#

root@control:~# openssl s_client -connect localhost:5000
CONNECTED(00000003)
depth=0 C = US, ST = Unset, O = Unset, CN = www.example.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = US, ST = Unset, O = Unset, CN = www.example.com
verify error:num=27:certificate not trusted
verify return:1
depth=0 C = US, ST = Unset, O = Unset, CN = www.example.com
verify error:num=21:unable to verify the first certificate
verify return:1
(eventlet.wsgi.server): 2013-04-23 12:24:14,694 DEBUG wsgi write (32415) accepted ('127.0.0.1', 45734)

---
Certificate chain
 0 s:/C=US/ST=Unset/O=Unset/CN=www.example.com
   i:/C=US/ST=Unset/L=Unset/O=Unset/CN=www.example.com
---
Server certificate
-----BEGIN CERTIFICATE-----
MIICzTCCAjagAwIBAgIBATANBgkqhkiG9w0BAQUFADBXMQswCQYDVQQGEwJVUzEO
MAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEY
MBYGA1UEAxMPd3d3LmV4YW1wbGUuY29tMB4XDTEzMDQyMjE5MDc0N1oXDTIzMDQy
MDE5MDc0N1owRzELMAkGA1UEBhMCVVMxDjAMBgNVBAgTBVVuc2V0MQ4wDAYDVQQK
EwVVbnNldDEYMBYGA1UEAxMPd3d3LmV4YW1wbGUuY29tMIGfMA0GCSqGSIb3DQEB
AQUAA4GNADCBiQKBgQDo2haPef7LSbZWdjdahqE6OmNTKoLQfY2p5Kn5Yw/01vFF
NS+kDsj+ZJgFRihhS7VioxX+zOqGIBXIw38+D7u3pipERZ5+u5IxBbg88DsDw6BM
Azg0D+u1ZEpYblILM4xgr4/7OqChDSS73j5yHs/g2umklEE7Lnp0SGPezowsXwID
AQABo4G4MIG1MAkGA1UdEwQCMAAwHQYDVR0OBBYEFGnCB4LU/7rBshtrkqZ7osS4
NnnCMIGIBgNVHSMEgYAwfoAUdu2yRiw7296qR+K3eHQ4hsEli+OhW6RZMFcxCzAJ
BgNVBAYTAlVTMQ4wDAYDVQQIEwVVbnNldDEOMAwGA1UEBxMFVW5zZXQxDjAMBgNV
BAoTBVVuc2V0MRgwFgYDVQQDEw93d3cuZXhhbXBsZS5jb22CCQDmihg8bNUsPzAN
BgkqhkiG9w0BAQUFAAOBgQApd1twa0H8hl2VDDSVDBaUmLaKdXzDAmWcbcaE2YLD
cnbQm+RRfLvKQOtL5WcpaYdJrGkhg6QMny6a5M+GXGA6Y9/qbCqOj9vRE4gPAN6W
ItBHTdGtAQcOCxg2LP8FyRNeEApwA74SN1yM0xGj2UO0eRfuRg/YaF0ssxImaJ2U
OA==
-----END CERTIFICATE-----
subject=/C=US/ST=Unset/O=Unset/CN=www.example.com
issuer=/C=US/ST=Unset/L=Unset/O=Unset/CN=www.example.com
---
No client certificate CA names sent
---
SSL handshake has read 1058 bytes and written 440 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: zlib compression
Expansion: zlib compression
SSL-Session:
    Protocol : TLSv1.1
    Cipher : AES256-SHA
    Session-ID: F95160319636D97BFA4D7EB53E9A1CBA2E0D9219DE244F2FFF04AC00041BE994
    Session-ID-ctx:
    Master-Key: 17068D9DC7ED2F8EBCB240153AE7A2592ACC29803F3DCC36E8C49DE3C00C4026BDEED43D6B81DE9E0205E37219902A74
    Key-Arg : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 8a c4 ca 63 23 03 7d a0-ea 85 9a 28 37 98 49 1e ...c#.}....(7.I.
    0010 - e4 35 16 8d b0 19 7b df-42 17 94 f4 47 3e ab 55 .5....{.B...G>.U
    0020 - 6b 1d b6 07 9f 62 2b 7b-d0 83 38 82 cd 1f e4 f9 k....b+{..8.....
    0030 - 58 3a 2f 9c 0b 56 43 fe-40 8d 72 69 04 a3 f6 26 X:/..VC.@.ri...&
    0040 - e7 b4 b5 12 c6 52 98 92-a3 8b 3d af 7e 07 e7 7d .....R....=.~..}
    0050 - 0d 05 7f 3a 09 a4 75 21-34 d3 c8 8e 92 c8 bd 19 ...:..u!4.......
    0060 - 66 2e 73 ef 13 40 c8 76-63 20 11 b9 bc 3a da c6 f.s..@.vc ...:..
    0070 - 26 1c 08 48 b6 81 d1 a9-8c b3 6c 18 db dc 94 79 &..H......l....y
    0080 - c3 ae d5 bc 11 8e 48 cc-33 22 8e 75 2e 47 fd d5 ......H.3".u.G..
    0090 - 79 f2 a9 69 76 74 3e 47-f1 69 f9 8a b1 f2 08 17 y..ivt>G.i......

    Compression: 1 (zlib compression)
    Start Time: 1366745054
    Timeout : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)
---

read:errno=0
root@control:~#

Revision history for this message
xingzhou (xingzhou) said :
#3

In my env, when ssl is enabled, use HTTPS to visit, it worked as expected, but if I change back to visit by using HTTP, curl hangs, I'm using command like:

curl -k -H "X-Auth-Token:ADMIN" http://localhost:35357/v2.0/tokens/9e33ce48b9ade32258a62ccbbee7dc10

while using

curl -k -H "X-Auth-Token:ADMIN" https://localhost:35357/v2.0/tokens/9e33ce48b9ade32258a62ccbbee7dc10

is ok, is that possibly a bug?

Revision history for this message
Alfred Shen (alfredcs) said :
#4

My take is that if SSL is enabled then curl should go by https as a norm. The issue I am having now is that keystone returns "Authorization Failed" even with https. BTW from the token format it seems to me that you are on Folsom, unless the token configuration in keystone.conf had been tweaked. Right? Can you post your keystone.conf if possible?

Revision history for this message
xingzhou (xingzhou) said :
#5

here is my keystone.conf, I'm using devstack

[DEFAULT]
admin_token = ADMIN
log_dir = /var/log/keystone

[sql]
connection = mysql://root:010638@localhost/keystone?charset=utf8

[catalog]
driver = keystone.catalog.backends.sql.Catalog
[token]
driver = keystone.token.backends.sql.Token

[ec2]
driver = keystone.contrib.ec2.backends.sql.Ec2

[ssl]
enable = True
certfile = /etc/keystone/ssl/certs/signing_cert.pem
keyfile = /etc/keystone/ssl/private/signing_key.pem
ca_certs = /etc/keystone/ssl/certs/ca.pem
#key_size = 1024
#valid_days = 3650
#ca_password = None
cert_required = False
#cert_subject = /C=US/ST=Unset/L=Unset/O=Unset/CN=localhost

[signing]
token_format = PKI
#token_format = PKI
certfile = /etc/keystone/ssl/certs/signing_cert.pem
keyfile = /etc/keystone/ssl/private/signing_key.pem
ca_certs = /etc/keystone/ssl/certs/ca.pem
key_size = 1024
valid_days = 3650
ca_password = None
cert_subject = /C=US/ST=Unset/L=Unset/O=Unset/CN=www.example.com

[auth]
methods = password,token
password = keystone.auth.plugins.password.Password
token = keystone.auth.plugins.token.Token

[filter:debug]
paste.filter_factory = keystone.common.wsgi:Debug.factory

[filter:token_auth]
paste.filter_factory = keystone.middleware:TokenAuthMiddleware.factory

[filter:admin_token_auth]
paste.filter_factory = keystone.middleware:AdminTokenAuthMiddleware.factory

[filter:xml_body]
paste.filter_factory = keystone.middleware:XmlBodyMiddleware.factory

[filter:json_body]
paste.filter_factory = keystone.middleware:JsonBodyMiddleware.factory

[filter:user_crud_extension]
paste.filter_factory = keystone.contrib.user_crud:CrudExtension.factory

[filter:crud_extension]
paste.filter_factory = keystone.contrib.admin_crud:CrudExtension.factory

[filter:ec2_extension]
paste.filter_factory = keystone.contrib.ec2:Ec2Extension.factory

[filter:s3_extension]
paste.filter_factory = keystone.contrib.s3:S3Extension.factory

[filter:url_normalize]
paste.filter_factory = keystone.middleware:NormalizingFilter.factory

[filter:sizelimit]
paste.filter_factory = keystone.middleware:RequestBodySizeLimiter.factory

[filter:stats_monitoring]
paste.filter_factory = keystone.contrib.stats:StatsMiddleware.factory

[filter:stats_reporting]
paste.filter_factory = keystone.contrib.stats:StatsExtension.factory

[filter:access_log]
paste.filter_factory = keystone.contrib.access:AccessLogMiddleware.factory

[app:public_service]
paste.app_factory = keystone.service:public_app_factory

[app:service_v3]
paste.app_factory = keystone.service:v3_app_factory

[app:admin_service]
paste.app_factory = keystone.service:admin_app_factory

[pipeline:public_api]
pipeline = access_log sizelimit stats_monitoring url_normalize token_auth admin_token_auth xml_body json_body debug ec2_extension user_crud_extension public_service

[pipeline:admin_api]
pipeline = access_log sizelimit stats_monitoring url_normalize token_auth admin_token_auth xml_body json_body debug stats_reporting ec2_extension s3_extension crud_extension admin_service

[pipeline:api_v3]
pipeline = access_log sizelimit stats_monitoring url_normalize token_auth admin_token_auth xml_body json_body debug stats_reporting ec2_extension s3_extension service_v3

[app:public_version_service]
paste.app_factory = keystone.service:public_version_app_factory

[app:admin_version_service]
paste.app_factory = keystone.service:admin_version_app_factory

[pipeline:public_version_api]
pipeline = access_log sizelimit stats_monitoring url_normalize xml_body public_version_service

[pipeline:admin_version_api]
pipeline = access_log sizelimit stats_monitoring url_normalize xml_body admin_version_service

[composite:main]
use = egg:Paste#urlmap
/v2.0 = public_api
/v3 = api_v3
/ = public_version_api

[composite:admin]
use = egg:Paste#urlmap
/v2.0 = admin_api
/v3 = api_v3
/ = admin_version_api

Revision history for this message
adapaka bhavaniprasad (adapaka-prasad) said :
#6

Hi Alfred Shen ,

I am also facing same issue but it got resolved by giving the export SERVICE_ENDPOINT=http://10.10.56.19:35357/v2.0/
export SERVICE_TOKEN=ADMIN as a environment variables.

please paste your creds file here.

Regards,
Bhavani Prasad.

Revision history for this message
Koji (kj-tanaka) said :
#7

You guys would probably already have resolved this issue, but I leave some comment for people who will have the same issue.

Common Name is important for SSL. If the CN and the SERVICE_ENDPOINT are different, you will probably need to recreate your certificate with the same hostname + domain name. Something like CN=host.yoursite.org and SERVICE_ENDPOINT=http://host.yoursite.org:35357/v2.0/

Another good thing to know is, it looks Havana provides an easy way to setup SSL. Here's how I figured it out.

https://github.com/kjtanaka/havana_startup/wiki/How-to-enable-ssl-on-keystone

Can you help with this problem?

Provide an answer of your own, or ask Alfred Shen for more information if necessary.

To post a message you must log in.