PackStack for RHEL-OSP Revisited

Hi folks,

In today’s post we’re going to revisit our old standby tool, “PackStack”. Why? Because it’s worth remembering that it’s a solid tool to work with, even if it’s not very flexible. As much time as I spent on the RHEL-OSP Installer, what if you just want to stand up 2 or 3 nodes and be done with it? The Installer might be overkill. But then again, what if you want to duplicate what I did regarding the network prep in the Installer with PackStack?

Boom! That’s ~exactly~ what we’re doing today.

If you’re completely new to PackStack and OpenStack, then this is what you need to know. There are several ways to deploy OpenStack. Each distribution of OpenStack (RHEL-OSP, Mirantis, Piston, etc) has its own way(s). RHEL-OSP for example can be deployed manually via individual package, PackStack, RHEL-OSP Installer, Ansible, the upcoming Converged Installer, SpineStack, and several other methods. Each has its own merits and complexities. PackStack remains popular because it is simple. You either run it interactively (you answer questions) or you create a single answer file and then it goes off and deploys. Either way, it uses “Puppet” under the covers to deploy. That’s PackStack in a nutshell.

If you’re working with RHEL or CentOS, you’ll want to have RHEL 7 or CentOS 7 already deployed on 2 or 3 nodes. That is to say that 1 node will be a controller and the other node(s) will be compute nodes. Remember, PackStack doesn’t support HA deployments. And just like before, we’re working with 3 interfaces – deployment/mgmt, tenant/vxlan, and public/floating IP. In my specific environment, that turns out to be eno1, enp0s20u1, and enp0s20u2, respectively. (The 2nd and 3rd interface names look funky because they are actually USB to Ethernet dongles.) Be sure to configure /etc/hosts, /etc/resolv.conf, and your interface files!

So, we have our base operating systems deployed, and I’m going to assume that the systems are either registered and subscribed properly (RHEL 7) or simply subscribed (CentOS). I won’t go into detail as there are too many other resources covering this aspect and I want to spend more time on the PackStack config..

Next, we install PackStack:

# yum –y install openstack-packstack

Then create an “answer file”

# packstack --gen-answer-file=long-file.txt

Then back that file up:

# cp long-file.txt long-file.txt.bak

My friend and fellow hatter Mike W likes to work with a shorter, uncommented file (optional):

# cat long-file.txt | egrep –v “^$|#” > short-file.txt

 

You get to decide which file you want to edit. If you know what everything is, then you’re go with the short file. But if you’re still new to all of this, go with the long file. The comments are actually helpful and can help you remember what is what and what goes where.

 

Ok, so now assuming that your systems are installed and configured properly, we’ll edit the answer file of your choice. If you’ve edited PackStack before, you’re likely familiar with all of the services and IP addresses. Essentially, you say “y” to what you want and “n” what you don’t want:

  • For our purposes, say “n” to Swift, Sahara, Trove, Ironic, and Nagios.
  • Almost any service that has an IP address will be the deployment/mgmt IP of the controller node, in my case 192.168.200.7. The “CONFIG_COMPUTE_HOSTS” will be the compute nodes, in my case 192.168.200.8,192.168.200.9. See that there is no space between the 2 IP addresses.
  • Leave the VMware section alone.
  • As you roll through AMQP, MariaDB, Keystone, Glance, and Cinder, be sure that the IP address is for the controller node. Leave the passwords alone for now.
  • Assuming you have enough storage attached or locally available, change “CONFIG_CINDER_VOLUMES_SIZE=50G” to a larger size.
  • Leave the NetApp section alone.
  • When you get to NOVA and NOVA NETWORK, leave it alone. We’ll be working with Neutron Networking.
  • When you get to the Neutron section, here are the key/value pairs you’ll want to change:

CONFIG_NEUTRON_L3_EXT_BRIDGE=provider
CONFIG_NEUTRON_ML2_TYPE_DRIVERS=vxlan,flat
CONFIG_NEUTRON_ML2_VLAN_RANGES=physnet-external
CONFIG_NEUTRON_OVS_BRIDGE_MAPPINGS=physnet-external:br-ex
CONFIG_NEUTRON_OVS_BRIDGE_IFACES=br-ex:enp0s20u2
CONFIG_NEUTRON_OVS_TUNNEL_IF=enp0s20u1

Keep in mind, your interface names may be different than mine. Think of it this way:(eno1)eth0 -> deploy/mgmt -> 192.168.200.0/24 (data center only)
(enp0s20u1)eth1 -> tunnel/vxlan -> 192.168.100.0/24 (sdn/openstack only)(enp0s20u2)eth2 -> public/floating IP -> 10.0.1.0/24 (public)

In the “real world”, the public IP would not be a “10.0” address…

The “provider” network simply says we’re attaching the network to a physical network in the data center or in my case, my home lab. The VXLAN and Flat drivers get loaded so that we can use both VXLAN for tenant tunnels and “flat” (no tagging) for public traffic when we want. VXLAN is simply a better way of providing segmentation within SDN over traditional VLANs. “physnet-external” is an arbitrary name that is then used to map to “br-ex”, the External Bridge, that itself is associated with physical interface enp0s20u2.

Once you’re done editing, save it, then run PackStack against it:

# packstack --answer-file=(your answer-file)

Once it’s complete, it may have some networking (networks, subnets, router) setup. Delete them. Seriously, blow them away. The run the following script (pulled from one of my earlier posts), build_net.sh

 

#!/bin/sh

# Obviously, you'll want to change the values to suit your environment

PUB_NAME="Public"
PUB_SUB_NAME="Internet"
PUB_NW="10.0.1.0/24"
PUB_GW=10.0.1.1
PUB_START=10.0.1.60
PUB_END=10.0.1.100
PRIV_NAME="Private"
PRIV_SUB_NAME="PXE_Mgmt"
PRIV_NW="192.168.100.0/24"
PRIV_GW=192.168.100.1
PRIV_START=192.168.100.20
PRIV_END=192.168.100.100
RTR_NAME=Router01
ADMIN_TENANT_ID=ddc4d149e677432999d9bc3132b8e4f1
PHYS_NET=physnet-external

# sourch admin 
source /root/keystonerc_admin

# create the public network and subnet (FLAT)
neutron net-create $PUB_NAME --tenant-id $ADMIN_TENANT_ID --provider:network_type flat --provider:physical_network $PHYS_NET --router:external=True --shared
neutron subnet-create $PUB_NAME $PUB_NW --name $PUB_SUB_NAME --enable_dhcp=False --allocation-pool start=$PUB_START,end=$PUB_END --gateway=$PUB_GW

# create the private network and subnet
neutron net-create $PRIV_NAME
neutron subnet-create $PRIV_NAME $PRIV_NW --name $PRIV_SUB_NAME --allocation-pool start=$PRIV_START,end=$PRIV_END --gateway=$PRIV_GW

# create the router, interface for the private subnet, and gateway for the public subnet
neutron router-create $RTR_NAME
neutron router-interface-add $RTR_NAME $PRIV_SUB_NAME
neutron router-gateway-set $RTR_NAME $PUB_NAME

# restart the OpenStack Neutron services 
systemctl restart neutron-l3-agent.service
systemctl restart neutron-server.service

 

This will set your networks up just like we had them with the RHEL-OSP Installer in the posts from a few weeks ago.

Hope this helps!

Captain KVM

PS – Complete PackStack Answer File (reference only)

[general]
CONFIG_SSH_KEY=/root/.ssh/id_rsa.pub
CONFIG_DEFAULT_PASSWORD=
CONFIG_MARIADB_INSTALL=y
CONFIG_GLANCE_INSTALL=y
CONFIG_CINDER_INSTALL=y
CONFIG_NOVA_INSTALL=y
CONFIG_NEUTRON_INSTALL=y
CONFIG_HORIZON_INSTALL=y
CONFIG_SWIFT_INSTALL=n
CONFIG_CEILOMETER_INSTALL=y
CONFIG_HEAT_INSTALL=y
CONFIG_SAHARA_INSTALL=n
CONFIG_TROVE_INSTALL=n
CONFIG_IRONIC_INSTALL=n
CONFIG_CLIENT_INSTALL=y
CONFIG_NTP_SERVERS=10.0.1.1
CONFIG_NAGIOS_INSTALL=n
EXCLUDE_SERVERS=
CONFIG_DEBUG_MODE=n
CONFIG_CONTROLLER_HOST=192.168.200.7
CONFIG_COMPUTE_HOSTS=192.168.200.9,192.168.200.8
CONFIG_NETWORK_HOSTS=192.168.200.7
CONFIG_VMWARE_BACKEND=n
CONFIG_UNSUPPORTED=n
CONFIG_VCENTER_HOST=
CONFIG_VCENTER_USER=
CONFIG_VCENTER_PASSWORD=
CONFIG_VCENTER_CLUSTER_NAME=
CONFIG_STORAGE_HOST=192.168.200.7
CONFIG_SAHARA_HOST=192.168.200.7
CONFIG_USE_EPEL=n
CONFIG_REPO=
CONFIG_RH_USER=
CONFIG_SATELLITE_URL=
CONFIG_RH_PW=
CONFIG_RH_OPTIONAL=y
CONFIG_RH_PROXY=
CONFIG_RH_PROXY_PORT=
CONFIG_RH_PROXY_USER=
CONFIG_RH_PROXY_PW=
CONFIG_SATELLITE_USER=
CONFIG_SATELLITE_PW=
CONFIG_SATELLITE_AKEY=
CONFIG_SATELLITE_CACERT=
CONFIG_SATELLITE_PROFILE=
CONFIG_SATELLITE_FLAGS=
CONFIG_SATELLITE_PROXY=
CONFIG_SATELLITE_PROXY_USER=
CONFIG_SATELLITE_PROXY_PW=
CONFIG_AMQP_BACKEND=rabbitmq
CONFIG_AMQP_HOST=192.168.200.7
CONFIG_AMQP_ENABLE_SSL=n
CONFIG_AMQP_ENABLE_AUTH=n
CONFIG_AMQP_NSS_CERTDB_PW=PW_PLACEHOLDER
CONFIG_AMQP_SSL_PORT=5671
CONFIG_AMQP_SSL_CERT_FILE=/etc/pki/tls/certs/amqp_selfcert.pem
CONFIG_AMQP_SSL_KEY_FILE=/etc/pki/tls/private/amqp_selfkey.pem
CONFIG_AMQP_SSL_SELF_SIGNED=y
CONFIG_AMQP_AUTH_USER=amqp_user
CONFIG_AMQP_AUTH_PASSWORD=PW_PLACEHOLDER
CONFIG_MARIADB_HOST=192.168.200.7
CONFIG_MARIADB_USER=root
CONFIG_MARIADB_PW=b191c236517e4aac
CONFIG_KEYSTONE_DB_PW=96b7a25e802e433e
CONFIG_KEYSTONE_REGION=RegionOne
CONFIG_KEYSTONE_ADMIN_TOKEN=4ae59afcbf0b4f86a67b6a4dd2263fd4
CONFIG_KEYSTONE_ADMIN_PW=da0bf067ee8e49b8
CONFIG_KEYSTONE_DEMO_PW=2cec5633322644d9
CONFIG_KEYSTONE_TOKEN_FORMAT=UUID
CONFIG_KEYSTONE_SERVICE_NAME=keystone
CONFIG_GLANCE_DB_PW=6f00bdbcbf0b426b
CONFIG_GLANCE_KS_PW=7d8af8faa26d4e2a
CONFIG_GLANCE_BACKEND=file
CONFIG_CINDER_DB_PW=bfd8a3f482694a90
CONFIG_CINDER_KS_PW=a88c5d8bb71c4261
CONFIG_CINDER_BACKEND=lvm
CONFIG_CINDER_VOLUMES_CREATE=y
CONFIG_CINDER_VOLUMES_SIZE=50G
CONFIG_CINDER_GLUSTER_MOUNTS=
CONFIG_CINDER_NFS_MOUNTS=
CONFIG_CINDER_NETAPP_LOGIN=
CONFIG_CINDER_NETAPP_PASSWORD=
CONFIG_CINDER_NETAPP_HOSTNAME=
CONFIG_CINDER_NETAPP_SERVER_PORT=80
CONFIG_CINDER_NETAPP_STORAGE_FAMILY=ontap_cluster
CONFIG_CINDER_NETAPP_TRANSPORT_TYPE=http
CONFIG_CINDER_NETAPP_STORAGE_PROTOCOL=nfs
CONFIG_CINDER_NETAPP_SIZE_MULTIPLIER=1.0
CONFIG_CINDER_NETAPP_EXPIRY_THRES_MINUTES=720
CONFIG_CINDER_NETAPP_THRES_AVL_SIZE_PERC_START=20
CONFIG_CINDER_NETAPP_THRES_AVL_SIZE_PERC_STOP=60
CONFIG_CINDER_NETAPP_NFS_SHARES_CONFIG=
CONFIG_CINDER_NETAPP_VOLUME_LIST=
CONFIG_CINDER_NETAPP_VFILER=
CONFIG_CINDER_NETAPP_VSERVER=
CONFIG_CINDER_NETAPP_CONTROLLER_IPS=
CONFIG_CINDER_NETAPP_SA_PASSWORD=
CONFIG_CINDER_NETAPP_WEBSERVICE_PATH=/devmgr/v2
CONFIG_CINDER_NETAPP_STORAGE_POOLS=
CONFIG_IRONIC_DB_PW=PW_PLACEHOLDER
CONFIG_IRONIC_KS_PW=PW_PLACEHOLDER
CONFIG_NOVA_DB_PW=d65b23ea84eb4af5
CONFIG_NOVA_KS_PW=e5488fffaebf4637
CONFIG_NOVA_SCHED_CPU_ALLOC_RATIO=16.0
CONFIG_NOVA_SCHED_RAM_ALLOC_RATIO=1.5
CONFIG_NOVA_COMPUTE_MIGRATE_PROTOCOL=tcp
CONFIG_NOVA_COMPUTE_MANAGER=nova.compute.manager.ComputeManager
CONFIG_NOVA_COMPUTE_PRIVIF=eno1
CONFIG_NOVA_NETWORK_MANAGER=nova.network.manager.FlatDHCPManager
CONFIG_NOVA_NETWORK_PUBIF=enp0s20u2
CONFIG_NOVA_NETWORK_PRIVIF=eno1
CONFIG_NOVA_NETWORK_FIXEDRANGE=192.168.100.0/24
CONFIG_NOVA_NETWORK_FLOATRANGE=10.0.1.0/24
CONFIG_NOVA_NETWORK_AUTOASSIGNFLOATINGIP=n
CONFIG_NOVA_NETWORK_VLAN_START=100
CONFIG_NOVA_NETWORK_NUMBER=1
CONFIG_NOVA_NETWORK_SIZE=255
CONFIG_NEUTRON_KS_PW=8e211361d38b4e1e
CONFIG_NEUTRON_DB_PW=a39805421d1c48f6
CONFIG_NEUTRON_L3_EXT_BRIDGE=provider
CONFIG_NEUTRON_METADATA_PW=31012a13f5884dd7
CONFIG_LBAAS_INSTALL=n
CONFIG_NEUTRON_METERING_AGENT_INSTALL=n
CONFIG_NEUTRON_FWAAS=n
CONFIG_NEUTRON_ML2_TYPE_DRIVERS=vxlan,flat
CONFIG_NEUTRON_ML2_TENANT_NETWORK_TYPES=vxlan
CONFIG_NEUTRON_ML2_MECHANISM_DRIVERS=openvswitch
CONFIG_NEUTRON_ML2_FLAT_NETWORKS=*
CONFIG_NEUTRON_ML2_VLAN_RANGES=physnet-external
CONFIG_NEUTRON_ML2_TUNNEL_ID_RANGES=
CONFIG_NEUTRON_ML2_VXLAN_GROUP=
CONFIG_NEUTRON_ML2_VNI_RANGES=10:100
CONFIG_NEUTRON_L2_AGENT=openvswitch
CONFIG_NEUTRON_LB_INTERFACE_MAPPINGS=
CONFIG_NEUTRON_OVS_BRIDGE_MAPPINGS=physnet-external:br-ex
CONFIG_NEUTRON_OVS_BRIDGE_IFACES=br-ex:enp0s20u2
CONFIG_NEUTRON_OVS_TUNNEL_IF=enp0s20u1
CONFIG_NEUTRON_OVS_VXLAN_UDP_PORT=4789
CONFIG_HORIZON_SSL=n
CONFIG_SSL_CERT=
CONFIG_SSL_KEY=
CONFIG_SSL_CACHAIN=
CONFIG_SWIFT_KS_PW=61d834aecc974380
CONFIG_SWIFT_STORAGES=
CONFIG_SWIFT_STORAGE_ZONES=1
CONFIG_SWIFT_STORAGE_REPLICAS=1
CONFIG_SWIFT_STORAGE_FSTYPE=ext4
CONFIG_SWIFT_HASH=d707960ec5004c12
CONFIG_SWIFT_STORAGE_SIZE=2G
CONFIG_HEAT_DB_PW=PW_PLACEHOLDER
CONFIG_HEAT_AUTH_ENC_KEY=452ca8e8b055446e
CONFIG_HEAT_KS_PW=PW_PLACEHOLDER
CONFIG_HEAT_CLOUDWATCH_INSTALL=n
CONFIG_HEAT_CFN_INSTALL=n
CONFIG_HEAT_DOMAIN=heat
CONFIG_HEAT_DOMAIN_ADMIN=heat_admin
CONFIG_HEAT_DOMAIN_PASSWORD=PW_PLACEHOLDER
CONFIG_PROVISION_DEMO=n
CONFIG_PROVISION_TEMPEST=n
CONFIG_PROVISION_TEMPEST_USER=
CONFIG_PROVISION_TEMPEST_USER_PW=d79101b9ac6c4ecf
CONFIG_PROVISION_DEMO_FLOATRANGE=10.0.1.0/24
CONFIG_PROVISION_CIRROS_URL=http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-disk.img
CONFIG_PROVISION_TEMPEST_REPO_URI=https://github.com/openstack/tempest.git
CONFIG_PROVISION_TEMPEST_REPO_REVISION=master
CONFIG_PROVISION_ALL_IN_ONE_OVS_BRIDGE=n
CONFIG_CEILOMETER_SECRET=dd6ffe2da5d349b5
CONFIG_CEILOMETER_KS_PW=ec663f506c8e4f34
CONFIG_CEILOMETER_COORDINATION_BACKEND=redis
CONFIG_MONGODB_HOST=192.168.200.7
CONFIG_REDIS_HOST=192.168.200.7
CONFIG_REDIS_PORT=6379
CONFIG_SAHARA_DB_PW=PW_PLACEHOLDER
CONFIG_SAHARA_KS_PW=PW_PLACEHOLDER
CONFIG_TROVE_DB_PW=PW_PLACEHOLDER
CONFIG_TROVE_KS_PW=PW_PLACEHOLDER
CONFIG_TROVE_NOVA_USER=admin
CONFIG_TROVE_NOVA_TENANT=services
CONFIG_TROVE_NOVA_PW=PW_PLACEHOLDER
CONFIG_NAGIOS_PW=3c0a5b2db90845cb

3 thoughts on “PackStack for RHEL-OSP Revisited”

  1. I’ve been trying to get packstack to work as described here, after having set up the network described here: http://captainkvm.com/2014/12/openstack-installer-for-rhel-osp/

    I have CentOS 7 all around, a careful network map, and about a mile of labelmaker tape.

    So clearly, I have modified the deployment pictured on the above blog post: http://captainkvm.com/wp-content/uploads/2014/12/Slide21.jpg In particular, I do not have the two hosts in the blue box labeled KVM server (tower). Additionally, I merged “cloud controller” and “net controller” into one physical machine with three NICs. The remainder of my cluster I want to become compute nodes, with two NICs.

    My biggest problem thus far has been networking. The compute nodes couldn’t see the outside world because they’re only connected to two private networks, and so packstack failed on the first “yum” operation. I used firewall-cmd and enabled masquerading on the public zone, assigning the other two nics to “tenant” and “trusted” zones. I then set up previously unneeded gateways and DNS on the compute nodes. Ok, fine, now everyone can ping google.

    packstack seems to hate firewalld. At the very least it was reloaded, wiping out my runtime masquerade and zone assignments. I’m now trying to make the masquerade permanent but the controller node silently logs this error in /var/log/messages:

    Jul 2 18:40:32 plume firewalld: 2015-07-02 18:40:32 ERROR: Unable to add [‘POST_public_allow’, ‘!’, ‘-i’, ‘lo’, ‘-t’, ‘nat’, ‘-j’, ‘MASQUERADE’] into ipv4 nat

    The upshot is I’m having a really hard time making an environment where the compute nodes are all on private network(s), but packstack still runs. Do you have any suggestions?

    1. Hi Bryce,

      Sorry for the delayed response; I’ve been heads down on some other stuff. I would switch from firewalld to the traditional iptables. Merging the cloud and net controllers is no big deal – if you move to the later posts, that’s actually what ends up happening anyway. The Controller ends up with 3 interfaces and the compute node(s) have 2. So all hosts have a tunnel interface and pxe/mgmt interface, but the controller also has the external interface (as it is also now the network node).

      On to packstack. The diagram that you’re referring to really only applies if you’re using the RHEL-OSP installer – and that includes setting up masquerading (most likely anyway). If you’re just going to use packstack (which is fine) you just install CentOS (or RHEL) on your nodes, then packstack on what will be your controller… But now that I go back and read your post, I see what you’re saying… it couldn’t pull down the openstack bits because it wasn’t connected to the external network… right? Gotcha.. you can in fact either configure 3 networks on all, then disconnect the 3rd (external network) on the compute nodes, or configure the controller node to masquerade (NAT) the pxe/mgmt network to the external network and enable ip forwarding on the controller… and make the controller ip the gateway for the compute nodes.. that way, when they look for packages it hits the controller node on the pxe/mgmt interface, gets forwarded to the external interface and BOOM, you get your packages..

      If you need tips on the masquerade, look at chapter 3.5 of the installer guide: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/6/html/Installer_and_Foreman_Guide/Configuring_a_Gateway.html

      If you don’t have an account, it’s free to set one up.

      hope this helps,

      Captain KVM

      1. Hi and thanks for your help! I have two nodes up and I’m tentatively bringing up a third.

        Just to follow up, I tried again using iptables instead of firewalld for NAT. Same problem. When packstack installed neutron on the headnode, NATting stopped. Starting again by temporarily adding a connection to the public internet worked. I did have to rename all the network interfaces serving the tenant network to be the same, as this seems to be a packstack requirement.

Agree? Disagree? Something to add to the conversation?