Page Builder Dark Magic Locked Forever

"Love Locks" on Pont Neuf

Page Builder’s for Drupal and WordPress, at first feel like magic, but the Light can soon turn to Darkness.

Web CMS like Drupal and WordPress save development time and this thinking leads to the magic of a Page Builder.

Does complexity and increased total cost of ownership follow this initial simplicity? Are you locked in forever with a Page Builder? Is this bait and switch?

Page Builder Magic with Drupal
Page Builder Magic with Drupal

Saving Time and Money With Page Builders Today

Busy, stressed, want to get a product to market?

Want a website launched quickly?

A web CMS like WordPress and Drupal, offer paid page builders like Divi (WordPress) and Glazed/Carbon (Drupal).  There are also free ones like Panels and Display Suite in Drupal and some others I’ve not used on WordPress.

The overall philosophy of WordPress and Drupal, with their “clickety click” web interface, lead a site owner or site builder, to buy into the page builder route.

Very recently I launched 3 commercial websites in 4 weeks, for a local publisher, with the very excellent WordPress Divi Builder / page builder and the themes Divi & Extra from Elegant Themes.

From an end-user, site builder, no-coding required perspective DIVI and to a lesser extent, the newer Extra theme, are really, really amazing.

In Drupal projects I have struggled with Drupal 7 free page builders like Panels/Panels-In-Place-Editor.  A more magical page builder is the Drupal Distribution Panopoly and the distributions based on it.

It’s genuinely a really amazing feeling to be able to build a site in a day with WordPress & Divi.

You quickly get to the “looks great, thanks for building the website” moment. With gorgeous and functional landing pages. You can build a website in minutes and hours versus weeks to months. But…

But, What About Tomorrow?

The DIVI page builder downside came quickly with a simple request to change the standard functionality of a core Divi Builder module.

Under deadline pressure I looked at the code for the proprietary plugin Divi Builder and the Divi theme, and was overwhelmed by the hundreds of thousands of lines of code.

Seriously, that’s a lot of code. So, what looked like a small change turned into a steep learning curve, with the prospect of a fairly significant programming project.

The light magic of the page builder UX had a dark magic code underbelly.

Any page builder time-saving, looked about to be lost into a coding project of ambiguous value. This created a catch-22 situation which was frustrating and unsatisfactory to all.

Experienced WordPress developers will be shaking their heads and saying “I told you so, page builders are dark magic, page builders are evil”.

No doubt on the other side, Divi experts will also be shaking their heads and saying “dude, its all on you, buy a plugin, or code a custom Divi module, you just haven’t got the skills”.

I started a couple of coding experiments specific to requests. But the Divi Builder code is complex and large, and there was no real budget for this, and it was ultimately wasted effort.

Divi Ecosystem

There is a blogosphere of excellent tutorials and recipes and hacks. There are a few independent after-markets for plugins for the Divi page builder.

You can relatively easily find fixes and extensions and themes. Many of these seem like good value and the ecosystem is real and the community seems great.

But we had clearly left the Divi / Extra magic sweet spot and moved into a different place not so obvious up front with page builders.

Page Builder Locked In Forever

Possibly the biggest downside of Divi, is the intermingling of content and structure and display.

This is the single biggest problem and goes against every tenant of good web content management. Data should be separated from structure and display. Not all mixed up in a huge page of shortcodes.

One way to avoid this, I think, was to develop custom post types, and custom fields. Then use after market Divi modules to extend the Divi or Extra theme to accommodate this. This was not obvious and was not easy.

Locked In By the Magic of Page Builders

Our choices where now;

  • The screamingly obvious choice was narrow the scope of customization and work within the golden handcuffs.
  • Or focus on more fruitful aspects of web publishing.
  • Maybe a heads-first dive into the deep-end of Divi customization.
  • Perhaps swap out or mix-in third party plugins into Divi theme.

High Trust Client-Developer Relationships

It’s a precious client that wants to customize and understands the process and the costs.

When a client has a realistic understanding i.e. this is usually paid discovery, and they have a good sense of process, and this is an investment in time and has risks.

There may well be rabbit holes of complexity. It might be necessary to back out of one line of research and development. It might be better to outsource to a specialist.

Everything is wonderful if there is a high-trust relationship with the client. They have the time and money, understanding, maturity and patience to invest in such paid discovery, or outsourcing, or change in general.

Not a high trust relationship? The client doesn’t have time or money? If they need to get a product to market? If the perception is “my business is failing because the standard Divi module doesn’t work how I want, therefore my website is broken”.

Up goes the Red Flag and now there looms a big question mark over the entire project and the choice of technologies along the way.

Project & Task Management as Contract

Then project management becomes a kind of contract for settling disagreements and if these  project management artifacts don’t exist, well then, Houston, we have a problem.

Startups and Proof of Concepts with Page Builders

On a self-funded start-up R&D project I worked on with a friend we had to own the cost of similar lessons but on Drupal.

The project was based on a Drupal premium theme, which was very similar to Divi, a company Sooperthemes, with premium themes such as Glazed and a page builder called Carbon.

As a paying customer of premium themes from Sooperthemes in the past. I re-discovered Carbon & Glazed when struggling with an ancestor of Panopoly, a Drupal 7, Panels based site. A Panels page-builder website built by a leading agency and and other developers for a government client.

Drupal Panels Page Builder Magic

As the webmaster for this site, which had almost 20 Drupal Content Types, almost 7,000 nodes and a nightly feed via Drush from a JSON API from a custom core enterprise Rails app. This enterprise architecture had numerous iterations and felt like an archaeological dig.

On request, we documented the Panels based Landing Page procedure, for this GPL Panels based Drupal 7 site, and it came to something like 20 steps. Screenshots with many steps, which where counter intuitive. It involved bouncing between different screens and adjusting taxonomies and panel pages etc.

Sure, you can use Drupal 7 Panopoly and its way better than this. But there was no appetite for spending lots of time and money migrating the Drupal 7 site into a new Panopoly setup. Certainly not with 20 content types and 7000 nodes.

WordPress Custom Post Types and Drupal Entities

Also Drupal 8 is the next big thing in the Dries kindgom and seems to be maturing somewhat.

WordPress Custom Post Types / Advanced Content Fields is also a strong contender. A migration to WordPress was tempting as a Drupal 7 to Drupal 8 change is also a migration and not an upgrade.

Custom build-outs in Rails and Angular quietly lurk in the shadows.

At this stage I searched around and discovered Glazed and Carbon by Sooperthemes. When I bought a limited license and sent a colleague, the PHP programmer on the project, his response was “this is the coolest thing I ever saw!”. And in many ways it was.

Being ambitious, and poor, and foolish, I wanted to do a sustainability project website in Drupal 7 with Glazed and Carbon, and decided and make it a rating site.

Drupal 7 is a mature ecosystem, and you get a huge range of modules for free. Glazed and Carbon added a really excellent page builder experience, kind of like Divi.

But I soon discovered that some Drupal 7 modules, like the rating modules are effectively in maintenance mode, and there are long standing bugs.

Also, another Drupal page building style module ecosystem snagged development. Display Suite has some annoyances that I couldn’t troubleshoot via the web interface. Also, the Views for the directory of the rating site where slow and needed a refactored but these where generated originally from the proprietary theme.

A strange netherworld in between

Again, I’d left the sweet, safe magical space of “Page Builders”. This time in Drupal 7 page builders, a combination of free Display Suite and premium/paid Glazed/Carbon.

I had stumbled, blinded by Page Builder Light Magic into a Dark place. A place where the surface simplicity of the Page Builder Magic UX had created monstrously complex code. It felt like frickin Fantasia.

So, after 3 weeks of nearly full time work, including content creation by a friend, I decided reluctantly, that I wanted to do a second version.

A new version with a custom simplified Bootstrap on vanilla Drupal 7. I cloned the site with Aegir, and switched themes to Bootstrap and borked the cloned site. There was too much intermingled proprietary and non proprietary code.

Conversion was going to be complex and time consuming.

A combination of all of the above factors meant a rebuild was on the horizon, not a simple switch in theme.

Click Click Click Click, Refresh,  Click Click Click Click, Refresh

One of the other unsatisfying things about the Divi and Glazed is that the constant click click click. You get this experience site building with Drupal and WordPress in general, but this no coding philosophy is taken to the next level with magical page builders.

Ultimately it hurts, you get RSI, Repetitive Strain Injury. I switch hands with my mouse, change chairs and computers. Buts its stressful. Its a case of too much of a good thing.

Capturing Page Builder Config in Code ?

It’s all configuration via the web interface.

Theoretically,  a lot of these changes could be captured in code, in Drupal 7 that would be Features, and in WordPress the Configuration Management plugins, which I haven’t tried.

But this would untested and involve quite a bit of trial and error and would be in the shadow land between paid and free.

Open questions are:

  • Can I share the code and get community support?
  • Does this break the license if you forked parts of the code and make extensions and share the code?

With completely free i.e. GPL code you would have no issues.

Presumably there would be ways to deal with this.

Straddling is Not a Good Strategy

But the key thing is, you’ve left one business model i.e. paid premium easy page builder.

You are now straddling premium and free and it’s complex.

The overall cost of ownership becomes more than choosing one or the other and working within those constraints.

Surely there are experts in this hybrid of paid/free page builders. They must make a great living doing consulting and selling plugins.

Maybe its an easy fix with a plugin or two from the Divi after market, then again I am not so sure. I wonder how much of the experts work is saving projects from this messy straddling two worlds zone. There must be a lot of client therapy involved.

Proprietary code can be open source. Gitlab Enterprise hosts its proprietary code in public and enforces license payments via the admin UX. So, I assume it would be possible to publicly develop extensions of the code and get community AND company support. Is open source but with a proprietary license. TODO research this.

RMS would not be pleased.

Frustrated, searching the web, I read a blog piece about WordPress Divi, saying, its a great tool, but you better make sure when you use it for a site, its forever.

This article of course was suggesting a competing product and describing it middle ground solution called Beaver Builder. Maybe this was an affiliate link, maybe it was a genuine recommendation AND an affiliate link.

I’ve also had Beaver Builder recommended by two people personally. Beaver Builder has a page builder plugin, AND a Beaver Builder theme, but also advertises the use of ANY THEME.

Alternatives Today

So, this is where I am at currently, I am in the process of archiving my Glazed/Carbon development sites and didn’t renew.

The new version for rich page, rich design, landing page driven marketing sites, does look amazing. Some of the designs look cliched and like every other rich-design, rich-page marketing site out there on the web today. Fashions eh?

Owning the starter license for Divi, but a bit burned out and struggling to think of a site I will use it on. I don’t really want to do more work with it, and am unsure if I could recommend it to a client.

Excellent for a 5 to 10 page site that gets built in a day or a week and has only marginal updates. But if customization or migration becomes an requirement, well its a highly specialized area. It’s an entirely different proposition to the original super easy page builder proposition.

The same with Sooperthemes and Glazed? There is a short term gain, and then possibly you are locked in and its more expensive overall if you want to do advanced work. If I am wrong, please post comments.

A kind of bait and switch?

Super easy and almost free, and cheaper for the beginning stages. But then locked in and super complex on customization. I guess enough websites owners are okay with this, enough to not create a huge backlash.

No doubt I am wrong and there are talented experts out there who will share examples of how to completely avoid all these snares.

Horses For Courses

Plenty of marketing websites are basically traded in and scrapped every couple of years. And if the cost is low, like thousands versus 10,000’s than Divi and Glazed are still good value.

God knows web design is driven by fashions. i.e. scrap those text animations please, remove those rotating gauges please, take off the full width sliders and the luminescent primary colors please.

Anyways, enough vanity griping and verbose monologue.

I am building a hyperlocal startup site for myself with Beaver Builder now and I’ll let you know how I get on.

Do you love the Light Magic of Page Builders? Or are the Page Builders Dark Magic?

Aegir 3.8 on Ubuntu 16.04 Xenial on LXD LXC (eventually)

I have decided to write some books and work on apps from those. One book will be on website building and the other will be on self-hosting.

Therefore I am going to record the set-up, customization, programming and themes of various systems for hosting.

One I know relatively well is Aegir, basically Drupal hosting Drupal with Drush. I am going to work through Aegir 3.8 on Ubuntu Xenial with PHP 7 which is twice as fast.

As a second step I use the Remote Import feature to import sites from an OVH server.

The plan is to build a Drupal 7 and a Drupal 8 site each using my emerging framework.

Community Organizing

For a maybe 12 months I volunteered as community organizer for the Aegir Project and co-founded and co-organized an event at the United Nations hosted within nyccamp and opencamps.org.

This event is called Aegir Summit and we had Richard Stallman of Free Software Foundation give the Aegir group a strategy session and he also gave the entire summit a presentation on Free Software. The 2015 Aegir Summit archive site has details on sessions etc and the 2016 Aegir Summit featured NASA hosting with Aegir by Mobomo.

The Aegir Summit mini site has some additional info as does the marketing site Aegir Hosting System and the community site. I also gave a talk at Stanford Drupal Camp (slides) and at SCALE as report-back sessions from the New York event, basically telling my story and advocating commercial free software.

Virtual Machine

Am working on LXD / LXC I use a partly pre-configured Xenial LXC container

launch a fresh LXC container

enter enter the container and change hostname, hosts and network

lxc exec aegir-devekko-io bash

update and upgrade the container

apt-update && apt-upgrade

edit the /etc/hostname

aegir-devekko-io

/etc/hosts files, I turn off ipv6 as I cant use DNS with ipv6 on my LAN and it creates issues


127.0.0.1 aegir.devekko.io aegir-devekko-io localhost
REDACTED aegir.devekko.io aegir-devekko-io localhost
10.1.10.115 aegir.devekko.io aegir-devekko-io localhost

/etc/network/interfaces.d/eth0.cfg

The primary network interface


auto eth0:0 eth0:1 
iface eth0:0 inet static 
address REDACTED/28 
gateway REDACTED

iface eth0:1 inet static 
address 10.1.10.115/24

dns-nameservers REDACTED 8.8.8.8 
dns-search devekko.io

 

Pre-requisites for Aegir 3.8

From the Aegir docs we first update and upgrade

apt update && apt upgrade

we then check DNS and hostname

uname -a
Linux aegir-devekko-io 4.4.0-57-generic #78-Ubuntu SMP Fri Dec 9 23:50:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
hostname -i
127.0.0.1 REDACTED 10.1.10.115

install utilities

apt install git wget rsync

According to Aegir maintainer colan in response to my ISSUE on Drupal.org he does this for the Postfix MTA, I’ve been caught on this numerous times on Ubuntu and Aegir

apt install postfix bsd-mailx mutt

in Postfix config I choose Internet site and aegir.devekko.io as email domain

alter /etc/postfix/main.cf

inet_interfaces = 127.0.0.1

Install Aegir 3.9

add project repositories

echo "deb http://debian.aegirproject.org stable main" | sudo tee -a /etc/apt/sources.list.d/aegir-stable.list</pre>

and archive key to keyring

wget -q http://debian.aegirproject.org/key.asc -O- | sudo apt-key add -
sudo apt-get update
sudo apt update

Install Aegir 3.9 Debian Packages

sudo apt-get install aegir3

manually enter a Mysql password (some of the Ansible installers handle this)

choose an Aegir UI domain, Aegir is installed

start the file wall

sudo ufw allow http
Rules updated
Rules updated (v6)
[email protected]:~# sudo ufw enable
Firewall is active and enabled on system startup

Sites | aegir.devekko.io - Chromium_002

Note, I did actually have issues and had to autoremove and purge postfix and tried to manually install Php etc

Installation did finally work after considerable troubleshooting. I would say that there is defintely something happening during install on Xenial that is not being handled properly.

Aegir is installed, and now its time to add a Drupal 7 and a Drupal 8 site for my project.

More in later blog posts

Self hosting the Drupal web content management system with Aegir

Serious Drupal hosting SaaS such as Pantheon starts at 25$USD per site, Acquia is something like 100$+ per site.

With the FLOSS Aegir Hosting System this cost per site can be greatly reduced. Of course total cost of ownership may still be high, when technical ability and Linux admin time are factored in.

Let’s self-hosted Aegir on an LXD/LXC container and migrate off an OVH VPS using Aegir’s built-in Remote Import feature.

I’ve worked with Ansible for Aegir in the past and there are a few Ansible roles and example playbooks on github.

As far as I can tell, none are official, so, I’ll use the Debian packages on Ubuntu Trusty 14.04 with Php5 from the official docs.

Php7 on Xenial is reportedly twice as fast and is supported, but I had errors and had no time to work through the issues.

UPDATE: I did actually get Aegir 3.9 with PHP7 on Ubuntu 16.04, but not without issues, I would say there are still issues

 

 

vi /etc/hosts

127.0.0.1 aegir.demo.io aegir-server localhost
94.84.199.93 aegir.demo.io aegir-server
10.1.10.125 aegir.demo.io aegir-server
::1 aegir.demo.io aegir-server ip6-localhost ip6-loopback

lets do an update and upgrade

sudo apt update && sudo apt upgrade

I also add the server to Debops inventory. I want unattended upgrades of Ubuntu security releases and firewall via ferm etc.

install Postfix, choose Internet site

sudo apt install postfix

 

add the Debian official repositories

echo "deb http://debian.aegirproject.org stable main" | sudo tee -a /etc/apt/sources.list.d/aegir-stable.list

 

add the archive key to the servers keyring

wget -q http://debian.aegirproject.org/key.asc -O- | sudo apt-key add - sudo apt-get update

 

install the Aegir 3 package

sudo apt install aegir3

 

the installation proceeds and you need to manually enter a mysql password (make it secure, record it in an encrypted password manager and avoid special characters)

verify URL of hostmaster frontend

re-enter mysql password

Aegir is installed and offers a login link, change username and password and email.

More parts to follow