Week 37

Published on Author malmLeave a comment

Adventures with Docker II: Linking Docker Containers

[avatar user=”malm” size=”small” align=”left” link=”file” /]

In last week’s blog I outlined how to create and destroy a virtual machine (VM) on the Digital Ocean cloud using Docker Machine running on Windows. This week I’ll outline how to create a stack of Docker containers on that VM in order to bring up a vanilla WordPress installation. Assuming your VM instance is up and accessible via Docker, your next step is to try a docker run on the official MySQL and WordPress images in turn thus:

C:\> docker run --name mydb -e MYSQL_ROOT_PASSWORD=xxxx -e MYSQL_DATABASE=wp -d mysql
C:\> docker run --name mywp --link mydb:mysql -p 8080:80 -d wordpress

It turned out that this didn’t work in my setup.  In order to find out why, you need to run a docker logs command to check the container logs:

C:\> docker logs mydb
... 
2015-09-10 22:34:13 62 [Note] InnoDB: Initializing buffer pool, size = 128.0M InnoDB: mmap(137363456 bytes) failed; errno 12 
2015-09-10 22:34:13 62 [ERROR] InnoDB: Cannot allocate memory for the buffer pool 
2015-09-10 22:34:13 62 [ERROR] Plugin 'InnoDB' init function returned error. 2015-09-10 22:34:13 62 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 
2015-09-10 22:34:13 62 [ERROR] Unknown/unsupported storage engine: InnoDB 
2015-09-10 22:34:13 62 [ERROR] Aborting

The log reveals that the MySQL container has run out of memory. From reading around, this seems to be a common problem.  The solution is to switch to using a low-memory database configuration. @charltones has a suitable forked Docker setup using MariaDB (MySQL fork) that can be used to build this image:

C:\> git clone https://github.com/charltones/docker-mariadb-lo
C:\> docker build --tag="mariadb-lo" docker-mariadb-lo
C:\> docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
mysql latest 6762f304c834 31 hours ago 283.5 MB
wordpress latest 2ea81e5584d9 3 days ago 512.3 MB
mariadb-lo latest f2485761e714 2 weeks ago 302.2 MB
mariadb latest 187668a0b2c7 3 days ago 302.2 MB

After building it, you can retry the earlier docker run recipe to create a low-memory database container and then check the IP of the VM:

C:\> docker run --name mydb -e MYSQL_ROOT_PASSWORD=xxxx -e MYSQL_DATABASE=wp -d mariadb-lo
C:\> docker run --name mywp --link mydb:mysql -p 8080:80 -d wordpress
C:\> docker-machine ip dodocker 
45.55.172.85

You should at this point be able to navigate to the default WordPress installation at URL http://45.55.172.85:8080 of course remembering to substitute with your corresponding IP address.

You can still experience memory issues on account of using Apache especially in a small Digital Ocean instance so it is worth switching from Apache to an alternative nginx-fpm based web stack to save even more memory.  The fpm here stands for “fastCGI process manager”.   There’s evidence to suggest a staggering figure of up to 40 million hits a day can be supported by this architecture at a total cost of as little as $10/month on Digital Ocean.

In order to switch, it is necessary to build the official wordpress-fpm Docker image and an nginx-fpm web server for it.  For the latter, I used another starting point fork provided by @charltones.   This snippet outlines how you build the corresponding Docker images:

C:\> git clone https://github.com/docker-library/wordpress.git
C:\> docker build --tag="wordpress-fpm" wordpress/fpm
C:\> git clone https://github.com/charltones/docker-nginx-fpm
C:\> docker build --tag="nginx-fpm" docker-nginx-fpm

Docker best practice is to have each container do one thing only, though that seems to be frequently ignored.  In this case we can build a three-layer stack holding the database, web and app tier by employing Docker linking and volume support.  In the following snippet, three linked containers are being run from the above images:

C:\> docker run --name mydb -e MYSQL_ROOT_PASSWORD=xxxx -e MYSQL_DATABASE=wp -d mariadb-lomem
C:\> docker run --name mywp-fpm --link mydb:mysql -d wordpress-fpm
C:\> docker run --name mynginx-fpm --link mywp-fpm:fpm -p 8080:80 --volumes-from mywp-fpm -d nginx-fpm

The first time this was tried, the wordpress-fpm container failed to start and supplied this error message: “docker cannot start container no such file or directory”.  Fixing that required changing CRLF line endings in a shell script copied over from Windows to Linux as per this StackOverflow post.  You need to remove the existing nginx-fpm container and image and rebuild it after switching to LF line endings as follows:

C:\> docker rm -f mynginx-fpm
C:\> docker rmi nginx-fpm
C:\> cd wordpress/fpm
C:\> scite dockerentrypoint.sh -> change CRLF to LF!!
C:\> cd ../.. -> back to root for wordpress 
C:\> docker build --tag="wordpress-fpm" wordpress/fpm
C:\> docker run --name mynginx-fpm --link mywp-fpm:fpm -p 8080:80 --volumes-from mywp-fpm -d nginx-fpm

Now, finally, you should be able to bring up the default WP installation by navigating to the same IP-dependent http://45.55.172.85:8080 URL as before in your browser.  Under the hood we now have a site built on three separate linked Docker containers in a single VM.

Entirely coincidentally, while writing this post, I was hit with an mandatory database update after the root WordPress installation for the blog auto-updated from version 4.3 to 4.3.1!   Luckily I had saved the page content text to a text file a moment before.   It served as another reminder, if any were needed, of the importance of having a backup-restore regime in place for your WordPress  workflow:

WPdbupdate

Taken together the posts this week and last have respectively walked readers through a full recipe for: a) creating a cloud VM and b) automating the construction of a stack of linked containers to hold a low-memory vanilla installation of WordPress within it.  The next step from here is to configure that stack to handle backup and restore requirements.

Apple

The Apple Watch Hermes is unveiled. New Apple TV

  • A few more discerning analysts were able to peer beyond the product buzz to examine the iPhone Upgrade Program, a new move that could prove to be far more significant than anything else they announced in the long run.  The program seems specifically designed to further demote the role of network operators within the Apple ecosystem and represents “really bad news” for the likes of AT&T and Verizon:

With this brilliant move, Apple also cleans the pipeline for customers who are intrigued by a new iPhone but don’t know what to do with their current one. You only have so many family members who are happy with a hand down. The returning iPhones are good for Apple too. They can be refurbished and sold to a different customer segment or recycled for parts and materials.

iPhone upgrade program

  • Benedict Evans also focussed on the Upgrade Program highlighting another potentially profound consequence a subscription model could create – an extended market for the returned refurbished iPhones:

Rather than making an actual cheaper iPhone model say in the $250-$350 range, Apple just carries on selling models from 1 and 2 years ago at lower prices.

  • For years pundits have regularly exhorted Apple to make cheaper iPhones.  With this ‘cheap iPhones by stealth’ strategy as Evans calls it, they just did it without actually having to make any phones.  When this particular penny drops with Android OEMs it will be the mobile equivalent of that infamous coffee cup moment in the Usual Suspects…

Devices and Manufacturers

  • Another would-be plucky smartphone contender spun out of the ashes of Nokia’s Meego efforts, Jolla, also seems to be struggling. This week COO and figurehead Marc Dillon stepped aside with The Register suggesting that maybe their best hope lies in being chosen as the “independent” smartphone technology platform of choice by the Russian authorities.
  • Nokia themselves meanwhile increasingly seem set to turn to Android as their smartphone platform if persistent rumours are to be believed.   Engadget showcased some leaked photos of the C1, alleged to be their first of any sort since the disastrous (for those involved) divestment of their phone division to Microsoft.

The Nokia C1

Marshall dished out shots of JD and put on punk rock performances. At one point, an executive on stage proclaimed the company was going to [verb beginning with F] the competition, in the [orifice beginning with A]. Whether that’s going to catch on like Tim Cook’s “We think you’re going to love it,” I’m not sure. I’ll admit, though, it made me instantly warm to Marshall.

Google and Android

Google Play is available in China, but reaches only 21 million of an estimated 800 million Chinese mobile users, Warman said. The primary app stores of Internet giants Qihoo, Tencent and Baidu account for two thirds of the market.  … These players are unlikely to give away that advantage.

AndroidVersions2

Apps, Services and Design

  • Byte seems to be the hot app of the moment.   Built by the creator of Vine, it further blurs the line between applications and the media they create.  It’s a little tricky to get your head around but worth sticking with if only to gain a broader understanding of where app interfaces may evolve next:

“THERE’S NO REASON A UI COULDN’T BE A PIECE OF MEDIA.”

  • However, changing user interface with every release of your app constitutes “messing with your users” rather than progress:

Security

  • In equal parts depressing and unsurprising, this post by lava lamp analyses the contents of “paste as a service” proposition Ghostbin and finds it full of dodge:

Services that offer anonymity are great, and they solve some very interesting problems. Even so, anonymity is a tool that is often sought after and used by people without good intentions. While the numbers I’ve crunched above may contain some amount of inaccuracy, the picture remains clear: a significant amount of the pastes shared on Ghostbin have illicit content.

Percent Categories

DevOps

  • A great presentation from Pete Cheslock on how the ability of an organisation to leverage the benefit of devops largely depends upon the culture and structure of the organisation.  Which helps explain why the Romans would be awesome at devops:

  • From the presentation a great quote from Deming should serve to remind those that lead where to look first for problems:

Deming

  • Apparently WinOps is a thing and having a London Meetup next week to talk about among other things “Chef, Puppet, Ansible, Docker etc.”  Wonder if they’ll also mention CRLF-LF gotchas.

WinOps

Digital Enterprise

  • Another great post from Asymco which seeks to understand why Enterprise IT has been so sidelined by the social media analytics and cloud (SMAC) revolution.  As usual he nails it:

The problem is that enterprises have a capital use and allocation model which is obsolete. This capital decision process assumes that capital goods are expensive, needing depreciation, and therefore should be regulated, governed and carefully chosen. The processes built for capital goods are extended to ephemera like devices, software and networking.

Asia

  • Excellent TechCrunch post on why South-East Asia, not the US or Europe, is now the crucible for disruptive mobile business models. It’s a development that has been fuelled by a combination of “no-tail” leapfrogging of Web1.0 paradigms, a mass jump to a smartphone first market and open embrace of eCommerce over ad-based revenue models.

“Chinese visitors tend to plan upcoming shopping trips two to three months in advance … and research online price differences between luxury goods bought in their home markets and abroad.   This kind of shopping arbitrage has led to the rise of third-party selling agents, known as “Daigou”, which is fuelling the practice of savvy shoppers placing orders abroad.”

  • Interesting WSJ profile of Wingtech, one of a group of Chinese ODMs that started out in the wild world of Shanzai phone manufacture but are becoming increasingly technically sophisticated, automated and discerning both in terms of the work they take on and the calibre of their customers.  Wingtech’s  customers include Xiaomi, Huawei, Lenovo and Micromax:

Like most contract manufacturers, Wingtech does a lot more than just manufacture devices. It works with clients to design products, providing lower costs due to its large scale than what its clients could achieve by doing it all in-house.

In the last few years, hardware startup activity in Shenzhen has increased dramatically. It is now comparable to the software innovation system in Silicon Valley.

Wearables and the Internet of Things

  • The Raspberry Pi now has an official 7 inch capacitive touch 800×480 display that can be powered through GPIO port.  It’s available courtesy of ModMyPi for just over £50.  Interestingly there’s also an unofficial 10 inch screen from Kano with integrated battery that might be more suitable for media display projects.

Another technique—and the most intriguing—is bulk analysis of EEG streams, without knowing what the people submitting it are doing or thinking. With enough data and powerful artificial intelligence analysis, the system starts to find chunks of brainwave activity that are common to all people. Essentially, it starts to pick out the individual words and phrases used in the language of the brain.

  • Would you let these guys hack your EEG streams?

Artificial Intelligence

If you are sitting at a desk, driving a taxi or carrying a hod, stop for a moment and ask: could a robot or machine do this job better? … The answer, unfortunately for you, is probably – yes.

Each of these products is an example of a defining trend of our networked age:  the outsourcing of common sense to gadgetry. A foundational level of human perception and competence is being mechanized through apps and online services. The more mediated our lives become, the more we rely on media to make sense of the world for us. …  There is a fixed amount of intelligence in the world, and every time we think we’re creating new intelligence, what we’re actually doing is just redistributing the intelligence that’s already there.

intelligence

Software Engineering and Management

  • This “programmer competency matrix” offers a way to assess where your dev capabilities sit.  I’m not sure whether doing this blog represents level 2 or level 3 competency within the provided taxonomy:

Competency

  • InfoQ interview the author of Go Kit, “an open source microservice toolkit that can be used to facilitate and standardise the creation of Go-based services within the modern enterprise application stack.”

The greatest engineers who have brought us this far in software grew up in an entirely different era of computing — packed inside rooms filled with terminals, carrying mental models of new algorithm-based programs. Newer generations of programmers are born into a more fragmented world.

MOOCs

  • Whatever became of MOOCs?   This sharply observed post in The Kernel suggests that they’ve fallen to earth somewhat since the lofty rhetoric of 2011 when they were trumpeted as representing a “revolution in education”.  In fact, business model realities have forced a re-evaluation of priorities and MOOCs have essentially morphed into an “Uber for education” providing top-up domain specific training those who already have degrees. It helps contextualise Udacity’s introduction of nanodegrees:

MOOCs were lambasted for having a high dropout rate; the average completion rate still hovers around 15 percent, a level that would be unacceptable for a traditional face-to-face college class. And when the demographics of “successful” MOOC students were scrutinized in one University of Pennsylvania study, it was discovered that 80 percent already had college degrees.

Startups

  • FirstRound on startup scaling.  A kind of exponential growth characteristic of funded outfits that is a curious bedfellow to the their frequently “small agile team” culture origins:

There’s a unique feeling of ambiguity, chaos and stress that comes with doubling or tripling your team every six months. If you don’t manage scaling proactively, you can end up in trouble.

  • Crowdsourcing has proved an incredibly successful way of funding project development with success rates in the 90% range.   However, according to this Medium post, the biggest challenge the model faces is ensuring sufficient meaningful supply given it is “so foreign to most organizations that few are actually willing to let the crowd model work for them.“

The democratization of the tools of production, combined with the scale of society and increasing amounts of free time, affords more people the opportunity to contribute to projects, in ways large and small, than we could have ever imagined.

  • TechCrunch on the need for VC to innovate itself into a shape driven by “investors who prioritize sectors that make a real-world impact increase their bottom lines.”  That includes greater geographic, racial and economic diversity.  Until that happens, the inconvenient truth behind the Silicon Valley hype will remain in place:

Most technology startups remain relevant only to the best-off in society, leaving out billions of people and trillion-dollar markets.

Culture and Society

  • Stumbled upon a Quora post by Barack Obama explaining why the US-Iran nuclear deal is a good one.  Quite aside from the politics it still somehow remains a surprise to see the US President engaging on a social question and answer site.
  • Sleep seems to have been getting a lot of coverage lately.  This contrarian Quartz post suggests it is a myth that we’re not getting enough of it or as the author somewhat drily puts it:

there is little reason to lose sleep worrying about not sleeping enough.

  • Staying on the theme of future past, an odd TNW opinion piece which suggests your future self “hates you” because you procrastinate and do things without considering the impact on you in years to come:

When I see someone smoking a cigarette that’s obviously someone who hates his future self so much he doesn’t mind if he gets cancer.

Leave a Reply