Debops or the NSA for complete devops solution to network security

bastion debops

Production Ready Configuration Management Suites

As far as I know, in the entire world, there are two open source, production-based, bespoke configuration management systems.

In other words, if you want to run a full data center, network, production system and you want a system that has been designed and built and TESTED in production already to work, you only have two options.

Sure you can make your own system and test everything together, you can make a bricolage of existing roles and custom roles and make it work. But there will always be a long tail of implementation issues.

One such project comes from the US federal government, the NSA, SIMP, is based on Redhat and uses Puppet. There is excellent documentation and its compliant with the highest security standards (docs).

The other comes from a medical university in Europe and is based on Debian. Being an Ubuntu user, I’ll start with the Debian system. The second system is called Debops, the Debian data center in a box.

Debops is a Debian Data Center in a Box

Debops is based on the concept of an Ansible controller, i.e. a desktop or laptop which acts as the controller for the entire network.

I am going to install a bastion server, a VMware based VM which will be encrypted and also act as the bastion server for the entire network. This way my laptop does not become the weakest link in the entire chain.

I am running the VMware bastion VM from the RAID drive, and will give it a static IP, from within the VM and also at the router. My Comcast business router has annoying IP reassignment settings that kick in at very unfortunate times, such as after a power out etc. I force them to stick with dedicated IP.

Bastion set-up

I installed the operating system on the server, configure bridged, static public and private IP, setup DNS with Cloudflare, VNC access and openssh-server. I also install Ubuntu Desktop, 1, because I have loads of RAM and 2 I want the flexibility of using VNC if needed. I know, shell purists roll their eyes now.

Install openssh-server, keygens, and then allow root login with keys, create a root password, and then connect locally and remotely ssh [email protected] This is super important as otherwise you will be locked out of ssh from remote and can only use VNC. Also when you run debops for the first time you debops adds root to the users for remote logins. I also had to su devekko to change from root to my main sudo user to run the debops python scripts.


ssh-copy-id [email protected]
ssh-copy-id [email protected]

Debops Install

Debops has excellent, if dense, docs, and we use the Install docs


sudo -H apt install python-pip

let’s also upgrade pip


sudo -H pip install --upgrade pip

Next we install Debops itself with PIP


sudo -H pip install debops

Install Ansible

Debops uses Ansible and we need that installed now, but let’s install the latest development version via GIT and run from source


cd /home/devekko
sudo apt install -y git wget curl
sudo apt install -y build-essential libssl-dev libffi-dev python-dev
sudo -H pip install paramiko PyYAML Jinja2 httplib2 six
git clone git://github.com/ansible/ansible.git --recursive
cd ./ansible
source /home/devekko/ansible/hacking/env-setup

now we have a Ansible running from source


ansible --version
ansible 2.3.0 (devel bd036c15e0) last updated 2017/02/19 16:06:37 (GMT -700)
config file =
configured module search path = Default w/o overrides

Starting Debops project

Now we begin our Debops project. Debops is designed to have separate projects for separate networks of servers and applications. From Getting Started installs the Debops roles, playbooks and some handy Python scripts for shortcuts to standard verbose Ansible


cd /home/devekko

debops-update
DebOps playbooks have not been found, installing into /home/devekko/.local/share/debops/debops-playbooks

for a total of 107 playbooks

Initiate Project


debops-init /home/devekko/devekko-io
Creating new DebOps project in /home/devekko/devekko-io ...

Inventory

We now add our bastion server to the inventory


cd devekko-io
vi ansible/inventory/hosts
[debops_all_hosts]
bastion ansible_connection=local

Now we run Debops on our Bastion server


debops
....

PLAY RECAP ******************************************************************************************
bastion : ok=218 changed=124 unreachable=0 failed=0

TASK: debops.dhparam : Generate Diffie-Hellman params on Ansible Controller - 224.43s
TASK: debops.apt : Update APT cache ------------------------------------ 19.12s
TASK: debops.apt_install : Install requested APT packages -------------- 13.19s
TASK: debops.core : Install required core packages --------------------- 12.88s
TASK: debops.sshd : Ensure OpenSSH support is installed ---------------- 10.40s
TASK: debops.ferm : Configure ip(6)tables rules ------------------------- 8.98s
TASK: debops.ferm : Ensure ferm is installed ---------------------------- 7.39s
TASK: debops.nullmailer : Install required packages --------------------- 6.48s
TASK: debops.auth : Install auth-related packages ----------------------- 5.56s
TASK: debops.atd : Install atd ------------------------------------------ 5.09s

now, Debops runs 107 playbooks and bootstraps your Bastion server as the Ansible controller

we now have Debops installed as a Bastion server which we can encrypt and backup and access from across the network

bastion debops
bastion debops

Phoenix and Elixir Development

GitKraken

I want to start my Phoenix Elixir FolkBot project and want to see if I can implement some of the new features today.

I should be simply using the latest release and upgrading them, and I might revert, but lets see what we can learn now.

Fork the Phoenix Framework repo

I had the honor of being the 1000th fork of the Phoenix Framework, Phoenix project and I will use Gitkraken to review the changes and JetBrains IDEA to review the code in some detail.

Gitkraken

Based on the Electron IDE, which is also the basis of the Atom IDE, Gitkraken is a cross-platform GIT code review tool.

Gitkraken is perfect for moments like this where there are a few branches and and I want to understand the significant commits.

GitKraken
GitKraken

we can select a branch and see on the right side what changes where made to what files

GitKraken
GitKraken

We can see in this commit abc685656ab37f4a9ec8502f91ab406f422cbce7 that a bunch of big changes occurred “Add phx.gen.html and phx.gen.schema”.

GitKraken
GitKraken

JetBrains IDEA

Now we have a sense of what files we want to review, lets install JetBrains IDEA.

We install the JetBrains Toolbox app to allow us to install and update applications.

JetBrains Toolbox
JetBrains Toolbox

Once Jetbrains IDEA is installed we open the Phoenix GIT repo which is checked out to the “cm-phx-gen-next” branch.

We install the Elixir Plugins, restart and now we can view some of the files and changes.

phx_new.ex
phx_new.ex

Some New Files

The new files we are interested in are


phoenix/lib/mix/phoenix/context.ex
phoenix/lib/mix/phoenix/schema.ex
phoenix/lib/mix/tasks/phx.gen.html.ex
phoenix/lib/mix/tasks/phx.gen.json.ex
phoenix/lib/mix/tasks/phx.gen.schema.ex

one of the key changes is the Schema and Context which is explained in the docs, the generators are chained and the generator with the largest scope is Mix.Tasks.Phx.Gen.Html


Generates controller, views, and bounded context for an HTML resource.

  mix phx.gen.html Accounts User users name:string age:integer

The first argument is the context name followed by
the schema module and its plural name (used for resources and schema).

The above generated resource will contain:

* a context module in lib/accounts.ex, serving as the API boundary
  to the resource
* a schema in lib/accounts/user.ex, with an `accounts_users` table
* a view in lib/web/views/user_view.ex
* a controller in lib/web/controllers/user_controller.ex
* a migration file for the repository
* default CRUD templates in lib/web/templates/user
* test files for generated context and controller features

another big, significant commit is fb5c340e351715ecd324d3d8ceb18b4f0c779d5a

anyways, lots to explore

Cover Your A#$e with Jira Project & Task Management: Desktop & Mobile

Jira In Motion Board

For business and personal, for projects and task management, I use the Australian company Atlassian’s industry standard Jira.

On those mornings when another cup of coffee just can’t remind me of my pressing tasks, there is nothing quite like pulling up the Project list and picking a task and moving it to In Progress.

Also, when you have those great ideas, CAPTURE them, NOW, dump every single idea into a Epic, Story, Task. Don’t wait for next week, it will be gone. I can’t tell you how many times, I’ve come back to a project list and said to myself, what a good idea, I completely forgot about all that.

Avoid Rabbit Holes of Complexity

It’s totally possible to get lost in all sorts of complexity and setup with Jira and it can become a huge time sink.

Presumably this is why products like Wrike, Asana, Trello (now purchased by Atlassian), Basecamp are growing also.

Basic Jira

There are very, very many screens and views available, the standard one allows browsing of tickets on the left and refresh on right

[TECH-52] tech shopping - Devekko JIRA - Google Chrome_002
[TECH-52] tech shopping – Devekko JIRA – Google Chrome_002

Kanban Boards

A super simple Kanban board interface is great for Agile type projects. Select a task for Development or In Progress and get it Done.

Kanban
Kanban

Structure

A more advanced, but super useful Atlassian marketplace paid plugin is called Structure. It has a great UX with intuitive shortcuts and allows creation of folders in which you can quickly add Tasks and Sub Tasks

Structure
Structure

Plain Tasks

Possibly the very simplest Task management plugins is the very aptly named Plain Tasks. Again keyboard shortcuts and a simple UX allow quick and easy creation of lists from a brain dump of issues.

Nothing quite like dumping every conceivable thought into a huge list when first brainstorming a project. Of course, prioritization and structure can follow, but the most important thing is to get that list down, capture those ideas. Tomorrow morning they WILL BE GONE.

Plain Tasks
Plain Tasks

Jira In Motion; Android App

The default Jira mobile interface is ready only and basically is rubbish.

A nice app for Android is Jira in Motion, its fremium and organized around boards. So you need your project in Agile Boards, either Scrum or Kanban for this app to work.

Jira In Motion Board
Jira In Motion Board
Jira In Motion Board DONE
Jira In Motion Board DONE

Project & Task Management

Having been a professional project manager on some quite large projects (and also small projects!), its vital to have a kind of living contract with yourself, developers, business and client etc.

A project and task management system is possibly the most important tool for making sure project and tasks are delivered as agreed.

Without these systems, you are basically in a he-said/she-said situation when it comes to the almost inevitable mis-understanding or dispute.

In those very, very, very rare instances where you have a high-trust environment, its easy to resolve.

If not high trust, then you are toast without a project and task system. Basically, you need to Cover Your Arse.

Project & Task management doesn’t have to be overly complex or time consuming and with Agile methods like Scrum and Kanban and tools like Jira in Motion you can manage business as well as technical projects.

WordPress Bootstrap 4 and Material Design with Understrap Child Theme

Brand Primary

I am trying to standardize on Bootstrap 4 and Material Design and am going to test the UnderStrap theme direct from Github and will use the UnderStrap child-theme for our FolkBot WordPress theme.

We create the WordPress site, salts, database, user and gitignore

We change to the themes folder and use ZIP to download the parent and child-theme and the child-theme will have its own name folkbot-understrap.

we add the themes to GIT, create a repo on Gitlab and push the entire site repo.


git add .
git status
git commit -am "Understrap and Folkbot Understrap"
git status
git remote add gitlab_com [email protected]:folkbot/folkbot_wordpress.git
git push gitlab_com master

we activate the theme using WP cli


wp theme activate understrap
Success: Switched to 'UnderStrap' theme.

we check the browser and we see with our biological eyes in our human heads we have an Understrap theme

Folkbot Understrap default
Folkbot Understrap default

we activate the new FolkBot Understrap child theme


wp theme list
+--------------------+----------+--------+---------+
| name | status | update | version |
+--------------------+----------+--------+---------+
| folkbot_understrap | inactive | none | 0.2.2 |
| twentyfifteen | inactive | none | 1.7 |
| twentyseventeen | inactive | none | 1.1 |
| twentysixteen | inactive | none | 1.3 |
| understrap | active | none | 0.5.7 |
+--------------------+----------+--------+---------+

wp theme activate folkbot-understrap/

Success: Switched to 'UnderStrap Child' theme.

Folkbot Understrap
Folkbot Understrap

All pretty standard so far, next we are going to try to bring in the Daemonite Material Design Bootstrap 4 library.

Lets try to put it /var/www/html/wp-content/themes/folkbot_understrap/sass


git clone https://github.com/Daemonite/material.git

We add this to git by deleting the material repo’s own .git git dot folder

Material repo
Material repo

Custom CSS

Understrap wants Nodejs, Bower, Gulp and Browser Sync


sudo apt-get install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties libffi-dev nodejs
cd
git clone https://github.com/rbenv/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
exec $SHELL
git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build
echo 'export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"' >> ~/.bashrc
exec $SHELL
rbenv install 2.4.0
ruby -v
gem install bundler
curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
sudo apt-get install -y nodejs
sudo apt-get install -y build-essential
ruby -v
sudo npm install -g bower
bower --version

We change directory and install via npm install

Gulp Watch

We change the gulpfile.js so it works for our DNS and in our folder we run gulp watch

Now in folkbot-understrap/sass/theme/_child_theme_variables.scss we can change our $brand-primary: #4c4d4a; and gulp watch updates and we get a new theme color

Material Design

Next, we want to integrate Material Design


// Material Design for Bootstrap 4
@import "material/assets/sass/material";

@import "theme/child_theme_variables"; // <--------- Add your variables into this file. Also add variables to overwrite Bootstrap variables here
@import "assets/bootstrap4";// <--------- Loads Bootstrap3 or Bootstrap4. Change from /bootstrap3 to /bootstrap4 Watch out! just for testing in the moment!
@import "../src/sass/understrap/understrap/understrap";// <-------- Loads the UnderStrap defaults. Just a few classes to incorporate BS in WP
@import "../src/sass/understrap/theme/contact-form7";// <-------- Loads some basic styling for contact form 7 plugin

//Optional files - If you don´t use the corresponding scripts/fonts comment em out
@import "assets/font-awesome"; // <------- Font Awesome Icon font
@import "assets/underscores"; // <------- Underscores media styles

// Any additional imported files //
@import "theme/child_theme"; // <--------- Add your styles into this file

to modify Understrap theme settings we need to bring variables across from the theme and place them into
folkbot-understrap/sass/theme/_child_theme_variables.scss


$brand-primary: #fe9ecd;

// Basics of a navbar
$navbar-height: 60px;
$navbar-margin-bottom: 0px;
$navbar-border-radius: 0px;

we can now add the kitchen sink from Bootswatch Bootstrap 4 and we can see Material Design Bootstrap 4

we also change material design folkbot-understrap/sass/material/assets/sass/variables/_colour.scss and other variables


$brand-color: $palette-grey-500 !default;
$brand-color-dark: $palette-grey-700 !default;
$brand-color-light: $palette-grey-100 !default;
$brand-text-color: $white-primary !default;
$brand-text-color-dark: $white-primary !default;
$brand-text-color-light: $black-primary !default;

$brand-color-accent: $palette-green-a200 !default;
$brand-color-accent-dark: $palette-green-a400 !default;
$brand-color-accent-light: $palette-green-a100 !default;
$brand-text-color-accent: $white-primary !default;
$brand-text-color-accent-dark: $white-primary !default;
$brand-text-color-accent-light: $black-primary !default;

Brand Primary
Brand Primary

Obviously, this is just the start

Next, Phoenix, Elm, Angular, Sketch, Android and Swift

Bootstrap 4 Material Design in GPL

Material Design Bootstrap 4

I am trying to focus on a single CSS framework for FolkBot and some implementations in distributions, binaries, products and SAAS.

There is a very excellent, non free, paid and premium theme, called Material Design for Bootstrap. It actually has a fremium kind of model. It has a well structured license and business model, BUT, none of the options allow embedding in an open source project, and since I want complete freedom I have chosen another implementation.

This Material Design for Bootstrap 4 by an Australian web agency called Daemonite. This seems the most active, if not the most popular, and since Bootstrap 4 is still in development I want a responsive team behind the codebase. It seems to have all the components and kitchen sink with great docs.

Material Design Bootstrap 4
Material Design Bootstrap 4

Folkbot Drupal 8 Bootstrap based Custom theme

Folkbot Pink

Like the web at large, the formerly Twitter Bootstrap base theme is now the de-facto standard, the go-to theme in Drupal.

The internet is increasingly being dominated by mega corporations, and the leading FLOSS projects are often coming from projects sponsored by them.

For instance the Zurb Foundation project is struggling and many of its ambitious branches are un-maintained, from the issues on GH it seems competition from Google is denying Zurb oxygen.

Drupal 8 Custom Theme

Let’s build a custom distribution for the Folk Bot Drupal 8 Distribution.

From the Bootstrap Drupal 8 Subtheming docs with the SASS Starterkit.

Change directory to our Installation profile directory

[code lang=text]
cd /var/www/folkbot_drupal8/profiles/folkbot/themes
[/code]

we copy the SASS starterkit folder and create a new folder

Bootstrap SASS starterkits
Bootstrap SASS starterkits

and we rename it folkbot

folkbot theme based on sass starter kit
folkbot theme based on sass starter kit

Rename and Customize

Next we rename the and customize

[code lang=text]
THEMENAME/THEMENAME.starterkit.yml becomes folkbot/folkbot.info.yml
THEMENAME/THEMENAME.libraries.yml becomes folkbot/folkbot.libraries.yml
THEMENAME/THEMENAME.theme becomes folkbot/folkbot.theme
[/code]

Inside our folkbot.info.yml file we make changes

[code lang=text]
core: 8.x
type: theme
base theme: bootstrap

name: 'FolkBot Bootstrap'
description: 'FolkBot Uses the Bootstrap framework Sass source files and must be compiled (not for beginners).'
package: 'Bootstrap'

regions:
navigation: 'Navigation'
navigation_collapsible: 'Navigation (Collapsible)'
header: 'Top Bar'
highlighted: 'Highlighted'
help: 'Help'
content: 'Content'
sidebar_first: 'Primary'
sidebar_second: 'Secondary'
footer: 'Footer'
page_top: 'Page top'
page_bottom: 'Page bottom'

libraries:
– 'folkbot_bootstrap/global-styling'
– 'folkbot_bootstrap/bootstrap-scripts'
[/code]

we also change the sub-theme config settings

THEMENAME/config/install/THEMENAME.settings.yml becomes folkbot/config/install/folkbot.settings.yml

and THEMENAME/config/schema/THEMENAME.schema.yml becomes folkbot/config/schema/folkbot.schema.yml

/var/www/folkbot_drupal8/profiles/folkbot/themes/folkbot/config/schema/folkbot.schema.yml is also changed

[code lang=text]
# Schema for the theme setting configuration file of the THEMETITLE theme.

folkbot.settings:
type: theme_settings
label: 'FolkBot settings'
[/code]

Enable Folkbot Bootstrap theme

We can now see our new FolkBot Bootstrap sub-theme

FolkBot Bootstrap
FolkBot Bootstrap

Now its enabled

FolkBot enabled
FolkBot enabled

We check the homepage and see we need to add logo, favicons and compile the SASS to get the needed CSS.

FolkBot before SASS compile
FolkBot before SASS compile

Install SASS Compiler

Now we focus on the SASS starterkit docs we need to install Ruby and SASS

Install Ruby

Lets use Gorails for Ruby installation docs

Ruby pre-requisites

[code lang=text]
sudo apt-get update
sudo apt-get install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties libffi-dev nodejs
[/code]

there are a few Ruby version managers, Gorails recommends rbenv

[code lang=text]
cd
git clone https://github.com/rbenv/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
exec $SHELL

git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build
echo 'export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"' >> ~/.bashrc
exec $SHELL

rbenv install 2.4.0
rbenv global 2.4.0
ruby -v
[/code]

Once Ruby is installed we shall have Ruby

[code lang=text]
ruby -v
ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-linux]
[/code]

<h2>Install Bundler</h2>

We need the Ruby package manager for gems, called Bundler

[code lang=text]
gem install bundler
Fetching: bundler-1.14.4.gem (100%)
Did you know that maintaining and improving Bundler and RubyGems.org costs more than $25,000 USD every month? Help us keep the gem ecosystem free for everyone by joining the hundreds of companies and individuals who help cover these costs: https://ruby.to/support-bundler
Successfully installed bundler-1.14.4
Parsing documentation for bundler-1.14.4
Installing ri documentation for bundler-1.14.4
Done installing documentation for bundler after 5 seconds
1 gem installed

[/code]

we now install SASS

[code lang=text]
gem install sass
Fetching: sass-3.4.23.gem (100%)
Successfully installed sass-3.4.23
Parsing documentation for sass-3.4.23
Installing ri documentation for sass-3.4.23
Done installing documentation for sass after 4 seconds
1 gem installed
[/code]

Now we need to download the Bootstrap SASS code into our new subtheme as described in the Drupal Bootstrap <a href="http://drupal-bootstrap.org/api/bootstrap/starterkits%21sass%21README.md/group/sub_theming_sass/8">docs</a>

[code lang=text]
/var/www/folkbot_drupal8/profiles/folkbot/themes/folkbot (master) $
git clone https://github.com/twbs/bootstrap-sass.git bootstrap
[/code]

now we start SASS compilation and polling, where it watches the SASS folder for changes and compiles to style.css

[code lang=text]
/var/www/folkbot_drupal8/profiles/folkbot/themes/folkbot (master) $ sass –watch sass:css
>>> Sass is watching for changes. Press Ctrl-C to stop.
directory css
write css/style.css
write css/style.css.map
[/code]

we can now do some changes to SASS and see our results

vi /var/www/folkbot_drupal8/profiles/folkbot/themes/folkbot_bootstrap/sass/_default-variables.scss

[code lang=text]
$gray-base: #0a0c16 !default;
$gray-darker: lighten($gray-base, 13.5%); // #222
$gray-dark: lighten($gray-base, 20%); // #333
$gray: lighten($gray-base, 33.5%); // #555
$gray-light: lighten($gray-base, 46.7%); // #777
$gray-lighter: lighten($gray-base, 93.5%); // #eee

$brand-primary: darken(#ca278e, 6.5%); // #337ab7
$brand-success: #52b820;
$brand-info: rgba(58, 115, 196, 0.33);
$brand-warning: #f0965f;
$brand-danger: #d959c0;
[/code]

and our crazy color scheme can be seen

Folkbot Pink
Folkbot Pink