Deploying Cloud Foundry on Virtualbox using BOSH CLI v2


The BOSH CLI v2 greatly enhances the interaction and deployment of Cloud Foundry and other BOSH releases. The new release, at time of writing still in beta, is now written in Golang, which allows it to be distributed by a simply binary without dependencies. There is even support for Windows planned, though binaries are currently not distributed. There are quite a few changes in the CLI commands, check them out here. The CLI now also includes a mechanism to bootstrap the BOSH Director, which was done previously by using bosh-init or for bosh-lite with a Vagrantfile.

This post will walk you through the required steps to deploy bosh-lite on Virtualbox and deploy Cloud Foundry. Bosh-lite is a single VM development environment which deploys bosh releases in garden containers.

Prerequisites

  • git
  • Virtualbox
  • 8GB memory recommended
  • Linux or Mac
    • Windows is coming
  • cf cli

BOSH CLI v2

First of all we need to download the latest BOSH CLI v2. You can find the latest version here.
After than we make the CLI globally accessible:

chmod +x ~/Downloads/bosh-cli-*
sudo mv ~/Downloads/bosh-cli-* /usr/local/bin/bosh

BOSH Director Deployment

We need the bosh-deployment repository which holds a collection of BOSH manifests. Furthermore we are creating a dirctory to hold some deployment specific configuration files.

git clone https://github.com/cloudfoundry/bosh-deployment ~/workspace/bosh-deployment
mkdir -p ~/deployments/vbox
cd ~/deployments/vbox

The next step is already to bootstrap the bosh-lite director to your local Virtualbox installation. You can modify the network information if you like.

bosh create-env ~/workspace/bosh-deployment/bosh.yml \
  --state ~/deployments/vbox/state.json \
  -o ~/workspace/bosh-deployment/virtualbox/cpi.yml \
  -o ~/workspace/bosh-deployment/virtualbox/outbound-network.yml \
  -o ~/workspace/bosh-deployment/bosh-lite.yml \
  -o ~/workspace/bosh-deployment/bosh-lite-runc.yml \
  -o ~/workspace/bosh-deployment/jumpbox-user.yml \
  --vars-store ~/deployments/vbox/creds.yml \
  -v director_name="Bosh Lite Director" \
  -v internal_ip=192.168.50.6 \
  -v internal_gw=192.168.50.1 \
  -v internal_cidr=192.168.50.0/24 \
  -v outbound_network_name=NatNetwork

Lastly we are creating an bosh environment alias and a few environmental variables to reduce the amount to type. You also want to recreate these variables if you create a new shell.

bosh -e 192.168.50.6 --ca-cert <(bosh int ~/deployments/vbox/creds.yml --path /director_ssl/ca) alias-env vbox

export BOSH_CA_CERT=$(bosh int ~/deployments/vbox/creds.yml --path /director_ssl/ca)
export BOSH_CLIENT=admin
export BOSH_CLIENT_SECRET=$(bosh int ~/deployments/vbox/creds.yml --path /admin_password)
export BOSH_ENVIRONMENT=vbox

Cloud Foundry Deployment

We are going to use cf-deployment for deploying Cloud Foundry. The repository is still in its early stages but allows us to easily deploy Cloud Foundry, using Diego, in one simple step. It looks like it will be the successor of cf-release and diego-release .

First we are cloning the repository:

git clone https://github.com/cloudfoundry/cf-deployment ~/workspace/cf-deployment
cd ~/workspace/cf-deployment

The next step is to upload the required bosh-lite stemcell., which is referenced at the bottom of cf-deployment.yml .
At the time of writing:

stemcells:
- alias: default
  os: ubuntu-trusty
  version: "3421.11"

Using bosh interpolate we can extract the stemcell version and upload it:

export STEMCELL_VERSION=$(bosh int ~/workspace/cf-deployment/cf-deployment.yml --path /stemcells/alias=default/version)
bosh upload-stemcell https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-trusty-go_agent?v=$STEMCELL_VERSION

We are using newer cloud config  model, which allows the separation of IaaS specific configuration from the deployment manifest.

bosh update-cloud-config ~/workspace/cf-deployment/iaas-support/bosh-lite/cloud-config.yml

The last step is to create and deploy our cloud foundry release. This may take long time, mostly depending on your internet connection.

bosh -d cf deploy ~/workspace/cf-deployment/cf-deployment.yml -o ~/workspace/cf-deployment/operations/bosh-lite.yml --vars-store ~/deployments/vbox/deployment-vars.yml -v system_domain=bosh-lite.com

To speed up deployment you can use precompiled releases by adding following operations file:

-o ~/workspace/cf-deployment/operations/use-compiled-releases.yml

If nothing failed, you have successfully deployed Cloud Foundry! Yey!

The last step is to create a local route, to be able to access the Cloud Foundry Environment from your host.

Linux:

sudo route add -net 10.244.0.0/16 gw 192.168.50.6

Mac:

sudo route add -net 10.244.0.0/16 192.168.50.6

Configure Cloud Foundry

First we need to log-in into cloud foundry. The default username is admin and the password can be found in the deployment-vars.yml , which was generated by bosh deploy  in the previous steps.
The following command will log you in and read the password from the file.

cf login -a https://api.bosh-lite.com --skip-ssl-validation -u admin -p $(bosh interpolate ~/deployments/vbox/deployment-vars.yml --path /cf_admin_password)

Lastly we have to create an organization and a space:

cf create-org cloudfoundry
cf target -o cloudfoundry
cf create-space development
cf target -o cloudfoundry -s development

You are now ready to deploy your first app!

Deploy to Cloud Foundry

I’ve created a very simple python application which can be used for testing.

git clone https://github.com/vchrisb/cf-helloworld ~/workspace/cf-helloworld
cd ~/workspace/cf-helloworld
cf push

You should now be able to access the app locally via http://cf-helloworld.bosh-lite.com .

To try out Docker  support, we first need to enable docker support and push a docker image.

cf enable-feature-flag diego_docker
cf push test-app -o cloudfoundry/test-app

Suspend Environment

Please do not reboot or shutdown your BOSH Director VM, it won’t come up properly again!

To be able to save your state, you can can suspend and resume your BOHS Director with following commands:

vboxmanage controlvm $(bosh int ~/deployments/vbox/state.json --path /current_vm_cid) savestate
vboxmanage startvm $(bosh int ~/deployments/vbox/state.json --path /current_vm_cid) --type headless

Delete Environment

Your BOSH environment can be removed with following command:

bosh delete-env ~/workspace/bosh-deployment/bosh.yml \
  --state ~/deployments/vbox/state.json \
  -o ~/workspace/bosh-deployment/virtualbox/cpi.yml \
  -o ~/workspace/bosh-deployment/virtualbox/outbound-network.yml \
  -o ~/workspace/bosh-deployment/bosh-lite.yml \
  -o ~/workspace/bosh-deployment/bosh-lite-runc.yml \
  -o ~/workspace/bosh-deployment/jumpbox-user.yml \
  --vars-store ~/deployments/vbox/creds.yml \
  -v director_name="Bosh Lite Director" \
  -v internal_ip=192.168.50.6 \
  -v internal_gw=192.168.50.1 \
  -v internal_cidr=192.168.50.0/24 \
  -v outbound_network_name=NatNetwork
rm ~/deployments/vbox/*

SSH into BOSH Director

If you would like to get a console on the BOSH Director, you can use following commands:

umask 077; touch ~/deployments/vbox/director_priv.key
bosh int ~/deployments/vbox/creds.yml --path /jumpbox_ssh/private_key > ~/deployments/vbox/director_priv.key
ssh jumpbox@192.168.50.6 -i ~/deployments/vbox/director_priv.key

Conclusion

It is straight forward and easy to deploy Cloud Foundry with BOSH CLI v2  and cf-deployment  to your localhost. The new cli greatly enhances the usability and deployment efforts.

Happy pushing!


Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

17 thoughts on “Deploying Cloud Foundry on Virtualbox using BOSH CLI v2

  • Avatar
    Gordon

    Thanks for this very handy guide. I found one minor mistake in the Configure Cloud Foundry section, the path to bosh interpolate should be ~/deployments/vbox/deployment-vars.yml instead of the cf-deployment path.

  • Avatar
    Alexander Widak

    Hi, thank you for the article – I tried it but I get always “Error: Timed out pinging to xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx after 600 seconds” for all vms during deploy step “Compiling packages” … do you have tips to pin down the error?
    I thought it is my environment: proxy and mitm but also with direct internet access there are the same errors …

  • Avatar
    Hrishikesh

    Hi Christopher Banck,

    Thanks for such great doc,
    one thing is updated in BOSH CLI-v2, that the password for login to cf is updated to field name cf_admin_password instead of uaa_scim_users_admin_password.

    So the command will be now :

    cf login -a https://api.bosh-lite.com –skip-ssl-validation -u admin -p $(bosh interpolate ~/deployments/vbox/deployment-vars.yml –path /cf_admin_password)

  • Avatar
    BaoYun

    Nice post.

    I followed all the steps with default settings, but failed when updating cf instances at last. The error messages are as below:

    17:39:06 | Updating instance api: api/7329016e-eb99-4793-bc8e-ef6360f6cd55 (0) (canary) (00:26:46)
    L Error: ‘api/0 (7329016e-eb99-4793-bc8e-ef6360f6cd55)’ is not running after update. Review logs for failed jobs: cloud_controller_ng, routing-api
    17:50:12 | Updating instance uaa: uaa/a8980f34-30cd-4d8c-9939-f8f71551d02e (0) (canary) (00:37:53)
    L Error: Action Failed get_task: Task 00f0f09b-dcad-4b78-7381-e9593e70c5f3 result: 1 of 1 post-start scripts failed. Failed Jobs: uaa.

    17:50:12 | Error: ‘api/0 (7329016e-eb99-4793-bc8e-ef6360f6cd55)’ is not running after update. Review logs for failed jobs: cloud_controller_ng, routing-api

    After checking, it looks like a uaa problem:
    uaa post-start.stderr.log :
    Failed to connect to 127.0.0.1 port 8989: Connection refused
    uaa monit.log:
    ‘uaa’ failed, cannot open a connection to INET[localhost:8989/healthz] via TCP

    Is there any advises?
    Thanks a lot.

  • Avatar
    Aayush Bhandari

    Thank you for the detailed guide. I believe there is a minor mistake in the cf deployment section. Path to cloud-config.yml should be workspace/cf-deployment/iaas-support/bosh-lite/cloud-config.yml.

    The new command will be:
    bosh update-cloud-config ~/workspace/cf-deployment/iaas-support/bosh-lite/cloud-config.yml

  • Avatar
    Deekshit

    Hi,

    Thanks for the detailed steps of installation, this has helped us a ton. Thank you so much.

    But one issue, after the cf deployment we are executing the command “cf marketplace” , but it is not listing down any services..

    Can you help me in getting the major services.

    Thanks,
    Deekshit

  • Avatar
    Ghafran Abbas

    Slight improvement to stemcell upload

    export STEMCELL_OS=$(bosh int ~/workspace/cf-deployment/cf-deployment.yml –path /stemcells/alias=default/os)
    export STEMCELL_VERSION=$(bosh int ~/workspace/cf-deployment/cf-deployment.yml –path /stemcells/alias=default/version)
    bosh upload-stemcell https://bosh.io/d/stemcells/bosh-warden-boshlite-${STEMCELL_OS}-go_agent?v=${STEMCELL_VERSION}

  • Avatar
    Ghafran Abbas

    Getting this error consistently during bosh cf deploy

    Task 31

    Task 31 | 13:06:48 | Preparing deployment: Preparing deployment (00:00:05)
    Task 31 | 13:07:01 | Preparing package compilation: Finding packages to compile (00:00:01)
    Task 31 | 13:07:02 | Creating missing vms: nats/7c8218d3-8566-4aeb-bc68-602480e6426a (0)
    Task 31 | 13:07:02 | Creating missing vms: adapter/f57be63d-cdab-484d-b343-311f7d01a54f (0)
    Task 31 | 13:07:02 | Creating missing vms: singleton-blobstore/371e7c08-e9bb-469c-82bb-a156eef76e15 (0)
    Task 31 | 13:07:02 | Creating missing vms: diego-api/5f2f4d54-5644-4dba-80ea-9d4698c7d413 (0)
    Task 31 | 13:07:02 | Creating missing vms: uaa/23b3ecc0-018c-4b6a-8285-6cf5afd6b86a (0)
    Task 31 | 13:07:02 | Creating missing vms: api/5777f8c6-a507-474c-8b36-ab47ba0010e2 (0)
    Task 31 | 13:07:02 | Creating missing vms: database/cb745bea-6700-4b14-bbba-95525fd02c81 (0)
    Task 31 | 13:07:02 | Creating missing vms: cc-worker/d2b999b8-182d-4c65-b21a-8810cbc05104 (0)
    Task 31 | 13:07:02 | Creating missing vms: scheduler/89981132-488b-41e6-a288-b2cbe262fb1b (0)
    Task 31 | 13:07:02 | Creating missing vms: router/581a27e9-5b5e-4956-8b16-efc91aa665f1 (0)
    Task 31 | 13:07:02 | Creating missing vms: tcp-router/3cf474b9-f36f-4901-8ba5-becc7cbd5444 (0)
    Task 31 | 13:07:02 | Creating missing vms: doppler/5707c1a4-328f-4eba-835e-6c7e6c75f914 (0)
    Task 31 | 13:07:02 | Creating missing vms: credhub/94a1adc7-ec62-43e2-899e-680b13098361 (0)
    Task 31 | 13:07:02 | Creating missing vms: diego-cell/be1f3be0-1583-4ead-aea3-88ea7132b7a0 (0)
    Task 31 | 13:07:02 | Creating missing vms: log-api/5a1d69e3-d7f6-452e-99cd-b840fbcb4990 (0) (00:00:55)
    Task 31 | 13:07:59 | Creating missing vms: uaa/23b3ecc0-018c-4b6a-8285-6cf5afd6b86a (0) (00:00:57)
    Task 31 | 13:08:00 | Creating missing vms: tcp-router/3cf474b9-f36f-4901-8ba5-becc7cbd5444 (0) (00:00:58)
    Task 31 | 13:08:00 | Creating missing vms: adapter/f57be63d-cdab-484d-b343-311f7d01a54f (0) (00:00:58)
    Task 31 | 13:08:01 | Creating missing vms: singleton-blobstore/371e7c08-e9bb-469c-82bb-a156eef76e15 (0) (00:00:59)
    Task 31 | 13:08:01 | Creating missing vms: diego-api/5f2f4d54-5644-4dba-80ea-9d4698c7d413 (0) (00:00:59)
    Task 31 | 13:08:01 | Creating missing vms: database/cb745bea-6700-4b14-bbba-95525fd02c81 (0) (00:00:59)
    Task 31 | 13:08:02 | Creating missing vms: scheduler/89981132-488b-41e6-a288-b2cbe262fb1b (0) (00:01:00)
    Task 31 | 13:08:02 | Creating missing vms: credhub/94a1adc7-ec62-43e2-899e-680b13098361 (0) (00:01:00)
    Task 31 | 13:08:02 | Creating missing vms: doppler/5707c1a4-328f-4eba-835e-6c7e6c75f914 (0) (00:01:00)
    Task 31 | 13:08:03 | Creating missing vms: nats/7c8218d3-8566-4aeb-bc68-602480e6426a (0) (00:01:01)
    Task 31 | 13:08:03 | Creating missing vms: router/581a27e9-5b5e-4956-8b16-efc91aa665f1 (0) (00:01:01)
    Task 31 | 13:08:03 | Creating missing vms: cc-worker/d2b999b8-182d-4c65-b21a-8810cbc05104 (0) (00:01:01)
    Task 31 | 13:08:03 | Creating missing vms: diego-cell/be1f3be0-1583-4ead-aea3-88ea7132b7a0 (0) (00:01:01)
    Task 31 | 13:08:04 | Creating missing vms: api/5777f8c6-a507-474c-8b36-ab47ba0010e2 (0) (00:01:02)
    Task 31 | 13:08:05 | Updating instance adapter: adapter/f57be63d-cdab-484d-b343-311f7d01a54f (0) (canary)
    Task 31 | 13:08:05 | Updating instance nats: nats/7c8218d3-8566-4aeb-bc68-602480e6426a (0) (canary) (00:00:40)
    Task 31 | 13:08:46 | Updating instance adapter: adapter/f57be63d-cdab-484d-b343-311f7d01a54f (0) (canary) (00:00:41)
    Task 31 | 13:08:46 | Updating instance database: database/cb745bea-6700-4b14-bbba-95525fd02c81 (0) (canary) (00:01:43)
    Task 31 | 13:10:29 | Updating instance diego-api: diego-api/5f2f4d54-5644-4dba-80ea-9d4698c7d413 (0) (canary) (00:20:14)
    L Error: ‘diego-api/5f2f4d54-5644-4dba-80ea-9d4698c7d413 (0)’ is not running after update. Review logs for failed jobs: bbs, silk-controller, locket
    Task 31 | 13:30:43 | Error: ‘diego-api/5f2f4d54-5644-4dba-80ea-9d4698c7d413 (0)’ is not running after update. Review logs for failed jobs: bbs, silk-controller, locket

    Task 31 Started Fri Oct 12 13:06:48 UTC 2018
    Task 31 Finished Fri Oct 12 13:30:43 UTC 2018
    Task 31 Duration 00:23:55
    Task 31 error

    Updating deployment:
    Expected task ’31’ to succeed but state is ‘error’

    Exit code 1

  • Avatar
    AJ

    Hi,
    The
    vboxmanage controlvm $(bosh int ~/deployments/vbox/state.json –path /current_vm_cid) savestate
    vboxmanage startvm $(bosh int ~/deployments/vbox/state.json –path /current_vm_cid) –type headless
    Is a great workaround if it works. the savestate leaves the VM in a Saved state. But.. when I want to start it again, I have the following error:
    Failed to open a session for the virtual machine vm-fc4fe56d-a93a-41bd-5a98-33b8454993a2.

    Failed to load unit ‘lsilogicscsi’ (VERR_SSM_LOADED_TOO_LITTLE).

    Result Code: NS_ERROR_FAILURE (0x80004005)
    Component: ConsoleWrap
    Interface: IConsole {872da645-4a9b-1727-bee2-5585105b9eed}

    So now I’m in a deadlock, unable to a delete-env, because the vm is in a wrong state. Any suggestions?