Trying out Openstack Function as a Service (Picasso)

Background

Function as a Service is a very exciting concept offering a scalable compute resource without having to worry about the underlying platform, and now it could be coming to an OpenStack powered cloud near you (well actually its still very early days, but I decided to have a play with it anyway).

For the last few years I have been running www.0870converter.co.uk completely “serverless” using AWS S3, API Gateway, Lambda and DynamoDB. The site is served up from an S3 bucket using the static website feature, this in turn reaches out to AWS API Gateway via JS when someone searches for a number (try 0800001066 if you don’t know any UK numbers) which invokes a Lambda function to run a small Python script to look up any alternative numbers in a DynamoDB table and the result is passed back through the microservices until the API Gateway returns the results to the async JS call. This comes at a great advantage to me, it means the website is fully scalable without any complex architectural decisions, I don’t have to manage any instances, any DBs, any services or apply any patches. I just push out code updates that do the interactive bits to Lambda and any static content / client-side JS to the S3 bucket. This is the power of Function as a Service, the only problem is until now the only really credible solution is Amazon Web Services as the ecosystem surrounding Lambda such as API Gateway open up great possibilities.

However recently there has been a flurry of development on a relatively new Openstack project, Picasso. Picasso wraps the IronFunctions Function as a Service engine in an Openstacky API including Keystone integration for multitenancy and user authentication and an extension to Python Openstack Client allowing easy administration. This opens up intruiging possibilities such as running your own Function as a Service in your own Openstack Private Cloud or potentially one day having Function as a Service included in some Openstack Public Clouds. Here are my first impressions of Picasso…

Installation

I performed installation on “RobCloud” a small Openstack playground I maintain for myself to try out stuff with Openstack. Installation was pretty simple first off I installed an IronFunctions host using the docker container approach as described in their dev docs at http://open.iron.io and then continued to install the Picasso service, unlike most Openstack services it is pretty picky about having the latest and greatest Python release so I installed Python 3.5.2 from Miniconda, installation was then as easy as installing the requirements via pip and then installing Picasso itself using it’s setuptools file. You can get the current Picasso source from https://github.com/openstack/picasso. It is still very early days for the project and unlike projects in the big tent there are no versioned releases or packages, however the master branch worked fine when I tested it on Valentines day 2017 :-).

So a few observations… unlike other Openstack services Picasso does not currently use Oslo.Config, Oslo.Log or Oslo.Db so configuration and behaviour is slightly different to what we have grown to expect from other Openstack services. With Picasso there is no config file as far as I can see and all the stuff the service requires is passed as cli opts, this is a little annoying as instead of editing a nice friendly config file you have to bodge your stuff into your systemd script. Likewise as Oslo.Db has not been used the service does not use SQLAlchemy which means the only backend DB option is MySQL, which will raise some eyebrows particularly from the PostgreSQL lovers in the Openstack community. However the project is still young and I have posted a wealth of blueprints to hopefully steer standardisation.

Usage

Anyway lets get on to using the service, I updated my Keystone service catalogue to include my new Picasso functions service and added the relevant endpoints. I then downloaded the Picasso client from https://github.com/openstack/python-picassoclient and installed it in the same virtualenv as where I run my Openstack Python client from… After completing this step I can now see an additional fn branch of the options in the Openstack client.

So what are these apps and routes things? Well these map identically to the apps and routes in IronFunctions, before you can run a function you must create an app and then create a route within the app which maps to your function. But do not be fooled, these are not real REST routes and you cannot build a proper API out on top of them like you can with AWS API Gateway in my opinion as it lacks support for stuff like variables in the URI, and the only verb used is POST. However having said that you can make these routes publically accessible if you set your function to be public and maybe this is a precursor to another Openstack project which translates these into RESTful endpoints. So let’s actually run some code…

First I need to create a new app to hold a route to my function.

openstack fn apps create demo
+-------------+--------------------------------------------------+
| Field       | Value                                            |
+-------------+--------------------------------------------------+
| description | App for project 797d956bab374df3ab33ca4ff603e032 |
| created_at  | 2017-02-14 23:03:39.856657                       |
| updated_at  | 2017-02-14 23:03:39.856683                       |
| project_id  | 797d956bab374df3ab33ca4ff603e032                 |
| config      | None                                             |
| id          | 6836c39f0109420593eb77a33e63702d                 |
| name        | demo-797d956bab374df3ab33ca4ff                   |
+-------------+--------------------------------------------------+

and then create a route within the app to launch our function. For the purposes of this blog post I am going to run the example hello world function from Iron.io, however functions are essentially just Docker containers hosted on DockerHub containing the code you wish to execute.

openstack fn routes create demo-797d956bab374df3ab33ca4ff /hello sync iron/hello
+-----------------+------------+
| Field           | Value      |
+-----------------+------------+
| image           | iron/hello |
| memory          | 128        |
| max_concurrency | 1          |
| timeout         | 30         |
| path            | /hello     |
| is_public       | False      |
| type            | sync       |
+-----------------+------------+

You can see from the command above you can define either synchronous or asynchronous functions, and “iron/hello” is the DockerHub image name. There are also options to set the max memory, concurrency and so on, however for this test I used the defaults. We can now execute the function as we can see the hello world example does indeed return hello world.

openstack fn routes execute demo-797d956bab374df3ab33ca4ff /hello
+---------+-------------------------------------------------------------------------+
| Field   | Value                                                                   |
+---------+-------------------------------------------------------------------------+
| message | App demo-797d956bab374df3ab33ca4ff sync route /hello execution finished |
| result  | Hello World!                                                            |
|         |                                                                         |
+---------+-------------------------------------------------------------------------+

There are a few oddities I noticed along the way, for example if you create an asynchronous function and kick it off via the API you never get an execution ID returned so you can later check in on progress, instead your function goes off and runs and you never get to see the return value, you have to implement callbacks yourself in the function to some other thing you manage like a DB or a queue and if they don’t work then it essentially goes and does its thing and you never get to know about it. Another issue is your Docker image must be published to DockerHub, I have not found a way to switch this out for a private Docker repository… It would be uber cool if it was extended to use Glance to store Docker images for use as functions and to eat the Openstack dog food as most Openstack clouds will already be running Glance for Nova images, so it makes sense to have all your images in one place. Other than these little niggles is does what it says on the tin, it runs functions. There is a whole host of additional functionality I can envisage such as Neutron tenant network integration, its all well and good being able to run functions, but if the thing executing your functions means it can’t talk to your database node on your private network you could be put in a situation of having MySQL or likewise listening publically. But the project is still young, I am hoping to get involved with building out some of the blueprints I have submitted following triage, so watch this space…

Public Demo

If you want to give it a go for yourself I decided (maybe stupidly) to put up a public demonstration up so you can fiddle with it. I have to warn you there is no monitoring and it’s something I put together pretty quickly so it will probably fall over from time to time and may be down for long periods without me checking in on it’s health. Also if it keeps falling over and requiring more attention than just a few service restarts here and there I’ll probably bin it off… I warn you, do not use this demo for anything serious, it may be down for long periods of time or dissappear without notice!

 

Identity URL: http://104.130.11.76:5000/v3/

Identity User: picasso_demo

Identity Password: d3m0t1m3 (Please do not change the password) – if people play around with this I’ll probably setup a cron to reset it every couple of minutes, let’s see what happens.

 

or if you’d prefer to copy and paste export the environment variables…

export OS_USERNAME=picasso_demo

export OS_PASSWORD=d3m0t1m3

export OS_PROJECT_NAME=picasso_demo

export OS_USER_DOMAIN_NAME=Default

export OS_PROJECT_DOMAIN_NAME=Default

export OS_AUTH_URL=http://104.130.11.76:5000/v3/

export OS_IDENTITY_API_VERSION=3

 

The easiest way to use the demo is to install the Openstack client, first I’d start by creating a virtualenv to seperate your Picaso demo activities from your usual Python work, then install Python Openstack Client followed by the Python Picasso Client as per below…

MRF28PG8WN:envs robe8437$ virtualenv picassoclient
Using base prefix '/Library/Frameworks/Python.framework/Versions/3.5'
New python executable in /Users/robe8437/Python/envs/picassoclient/bin/python3.5
Also creating executable in /Users/robe8437/Python/envs/picassoclient/bin/python
Installing setuptools, pip, wheel...done.
MRF28PG8WN:envs robe8437$ source picassoclient/bin/activate
(picassoclient) MRF28PG8WN:envs robe8437$ pip install python-openstackclient
(picassoclient) MRF28PG8WN:envs robe8437$ git clone https://github.com/openstack/python-picassoclient.git
Cloning into 'python-picassoclient'...
remote: Counting objects: 149, done.
remote: Total 149 (delta 0), reused 0 (delta 0), pack-reused 149
Receiving objects: 100% (149/149), 43.95 KiB | 0 bytes/s, done.
Resolving deltas: 100% (71/71), done.
Checking connectivity... done.
(picassoclient) MRF28PG8WN:envs robe8437$ cd python-picassoclient/
(picassoclient) MRF28PG8WN:python-picassoclient robe8437$ python setup.py install

Once the client has finished installed copy and paste the environment variable export statements from above and then run the following to check the client is working, it should return an Openstack Identity Token.

(picassoclient) MRF28PG8WN:python-picassoclient robe8437$ openstack token issue
+------------+----------------------------------+
| Field      | Value                            |
+------------+----------------------------------+
| expires    | 2017-02-15T01:13:15+0000         |
| id         | 5d56397c4f884c82b717a5ab038d5890 |
| project_id | bad2137765d046bf8101c61bdc374a16 |
| user_id    | 1ea44bf4f75147e6a6ba017871c0a67e |
+------------+----------------------------------+

Next providing nobody else playing with the demo has deleted it try running the hello world function…

(picassoclient) MRF28PG8WN:python-picassoclient robe8437$ openstack fn routes execute demo-bad2137765d046bf8101c61bd /hello
+---------+-------------------------------------------------------------------------+
| Field   | Value                                                                   |
+---------+-------------------------------------------------------------------------+
| message | App demo-bad2137765d046bf8101c61bd sync route /hello execution finished |
| result  | Hello World!                                                            |
|         |                                                                         |
+---------+-------------------------------------------------------------------------+

Providing nobody has broken the demo or removed the function this should work, now you are free to play with Picasso on my demo environment. As I previously requested, please do not change the password and if you can help it try not to delete the hello function :-).

3 comments on “Trying out Openstack Function as a Service (Picasso)

  1. -

    Sorry I deleted your hello fonction. Cheers

    • - Post author

      Looks like it’s still there to me 🙂


      (picassoclient) MRF28PG8WN:~ robe8437$ openstack fn apps list
      +--------------------------------+--------+--------------------------------------------------+
      | name | config | description |
      +--------------------------------+--------+--------------------------------------------------+
      | demo-bad2137765d046bf8101c61bd | None | App for project bad2137765d046bf8101c61bdc374a16 |
      +--------------------------------+--------+--------------------------------------------------+
      (picassoclient) MRF28PG8WN:~ robe8437$ openstack fn routes list demo-bad2137765d046bf8101c61bd
      +------+--------+------------+--------+---------+-----------------+-----------+--------+
      | type | path | image | memory | timeout | max_concurrency | is_public | config |
      +------+--------+------------+--------+---------+-----------------+-----------+--------+
      | sync | /hello | iron/hello | 64 | 30 | 10 | False | |
      +------+--------+------------+--------+---------+-----------------+-----------+--------+
      (picassoclient) MRF28PG8WN:~ robe8437$ openstack fn routes execute demo-bad2137765d046bf8101c61bd /hello
      +---------+-------------------------------------------------------------------------+
      | Field | Value |
      +---------+-------------------------------------------------------------------------+
      | message | App demo-bad2137765d046bf8101c61bd sync route /hello execution finished |
      | result | Hello World! |
      | | |
      +---------+-------------------------------------------------------------------------+
      (picassoclient) MRF28PG8WN:~ robe8437$

  2. -

    sorry you are right robert, I probably deleted lambda from AWS cloud catalog instead
    Cheers

Leave a Reply

Your email address will not be published. Required fields are marked *