Adventures with Docker II: Linking Docker Containers
In last week’s blog I outlined how to create and destroy a virtual machine (VM) on the Digital Ocean cloud using Docker Machine running on Windows. This week I’ll outline how to create a stack of Docker containers on that VM in order to bring up a vanilla WordPress installation. Assuming your VM instance is up and accessible via Docker, your next step is to try a docker run on the official MySQL and WordPress images in turn thus:
C:\> docker run --name mydb -e MYSQL_ROOT_PASSWORD=xxxx -e MYSQL_DATABASE=wp -d mysql C:\> docker run --name mywp --link mydb:mysql -p 8080:80 -d wordpress
It turned out that this didn’t work in my setup. In order to find out why, you need to run a docker logs command to check the container logs:
C:\> docker logs mydb ... 2015-09-10 22:34:13 62 [Note] InnoDB: Initializing buffer pool, size = 128.0M InnoDB: mmap(137363456 bytes) failed; errno 12 2015-09-10 22:34:13 62 [ERROR] InnoDB: Cannot allocate memory for the buffer pool 2015-09-10 22:34:13 62 [ERROR] Plugin 'InnoDB' init function returned error. 2015-09-10 22:34:13 62 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2015-09-10 22:34:13 62 [ERROR] Unknown/unsupported storage engine: InnoDB 2015-09-10 22:34:13 62 [ERROR] Aborting
The log reveals that the MySQL container has run out of memory. From reading around, this seems to be a common problem. The solution is to switch to using a low-memory database configuration. @charltones has a suitable forked Docker setup using MariaDB (MySQL fork) that can be used to build this image:
C:\> git clone https://github.com/charltones/docker-mariadb-lo C:\> docker build --tag="mariadb-lo" docker-mariadb-lo C:\> docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE mysql latest 6762f304c834 31 hours ago 283.5 MB wordpress latest 2ea81e5584d9 3 days ago 512.3 MB mariadb-lo latest f2485761e714 2 weeks ago 302.2 MB mariadb latest 187668a0b2c7 3 days ago 302.2 MB
After building it, you can retry the earlier docker run recipe to create a low-memory database container and then check the IP of the VM:
C:\> docker run --name mydb -e MYSQL_ROOT_PASSWORD=xxxx -e MYSQL_DATABASE=wp -d mariadb-lo C:\> docker run --name mywp --link mydb:mysql -p 8080:80 -d wordpress C:\> docker-machine ip dodocker 184.108.40.206
You should at this point be able to navigate to the default WordPress installation at URL http://220.127.116.11:8080 of course remembering to substitute with your corresponding IP address.
You can still experience memory issues on account of using Apache especially in a small Digital Ocean instance so it is worth switching from Apache to an alternative nginx-fpm based web stack to save even more memory. The fpm here stands for “fastCGI process manager”. There’s evidence to suggest a staggering figure of up to 40 million hits a day can be supported by this architecture at a total cost of as little as $10/month on Digital Ocean.
In order to switch, it is necessary to build the official wordpress-fpm Docker image and an nginx-fpm web server for it. For the latter, I used another starting point fork provided by @charltones. This snippet outlines how you build the corresponding Docker images:
C:\> git clone https://github.com/docker-library/wordpress.git C:\> docker build --tag="wordpress-fpm" wordpress/fpm C:\> git clone https://github.com/charltones/docker-nginx-fpm C:\> docker build --tag="nginx-fpm" docker-nginx-fpm
Docker best practice is to have each container do one thing only, though that seems to be frequently ignored. In this case we can build a three-layer stack holding the database, web and app tier by employing Docker linking and volume support. In the following snippet, three linked containers are being run from the above images:
C:\> docker run --name mydb -e MYSQL_ROOT_PASSWORD=xxxx -e MYSQL_DATABASE=wp -d mariadb-lomem C:\> docker run --name mywp-fpm --link mydb:mysql -d wordpress-fpm C:\> docker run --name mynginx-fpm --link mywp-fpm:fpm -p 8080:80 --volumes-from mywp-fpm -d nginx-fpm
The first time this was tried, the wordpress-fpm container failed to start and supplied this error message: “docker cannot start container no such file or directory”. Fixing that required changing CRLF line endings in a shell script copied over from Windows to Linux as per this StackOverflow post. You need to remove the existing nginx-fpm container and image and rebuild it after switching to LF line endings as follows:
C:\> docker rm -f mynginx-fpm C:\> docker rmi nginx-fpm C:\> cd wordpress/fpm C:\> scite dockerentrypoint.sh -> change CRLF to LF!! C:\> cd ../.. -> back to root for wordpress C:\> docker build --tag="wordpress-fpm" wordpress/fpm C:\> docker run --name mynginx-fpm --link mywp-fpm:fpm -p 8080:80 --volumes-from mywp-fpm -d nginx-fpm
Now, finally, you should be able to bring up the default WP installation by navigating to the same IP-dependent http://18.104.22.168:8080 URL as before in your browser. Under the hood we now have a site built on three separate linked Docker containers in a single VM.
Entirely coincidentally, while writing this post, I was hit with an mandatory database update after the root WordPress installation for the blog auto-updated from version 4.3 to 4.3.1! Luckily I had saved the page content text to a text file a moment before. It served as another reminder, if any were needed, of the importance of having a backup-restore regime in place for your WordPress workflow:
Taken together the posts this week and last have respectively walked readers through a full recipe for: a) creating a cloud VM and b) automating the construction of a stack of linked containers to hold a low-memory vanilla installation of WordPress within it. The next step from here is to configure that stack to handle backup and restore requirements.
- Apple had their big event September 9. TNW provided a handy summary of the key product announcements which can be further reduced to these four key headliners:
- Hermes Apple Watch – a genuine luxury collaboration and a snip at under £1000
- iPad Pro – a 12.9 inch iPad with stylus “Surface killer” predicted by a cartoonist 3 years ago.
- Apple TV – can now run apps and “integrates deeply” with Siri
- iPhone6s/6s Plus – with 3D Touch, a “Live Photos” feature that will eat your storage and a significantly updated iSight camera.
This comic is 3 years old; presented without comment pic.twitter.com/38PGAii4cY
— Alex Donaldson (@APZonerunner) September 9, 2015
- A few more discerning analysts were able to peer beyond the product buzz to examine the iPhone Upgrade Program, a new move that could prove to be far more significant than anything else they announced in the long run. The program seems specifically designed to further demote the role of network operators within the Apple ecosystem and represents “really bad news” for the likes of AT&T and Verizon:
With this brilliant move, Apple also cleans the pipeline for customers who are intrigued by a new iPhone but don’t know what to do with their current one. You only have so many family members who are happy with a hand down. The returning iPhones are good for Apple too. They can be refurbished and sold to a different customer segment or recycled for parts and materials.
- It’s likely to result in further margin upside for Apple as the carriers respond in the only way they can – aggressive pricing reductions. Which will, in turn, only serve to exacerbate a long term trend to “dumb pipe” status:
- Benedict Evans also focussed on the Upgrade Program highlighting another potentially profound consequence a subscription model could create – an extended market for the returned refurbished iPhones:
Rather than making an actual cheaper iPhone model say in the $250-$350 range, Apple just carries on selling models from 1 and 2 years ago at lower prices.
- For years pundits have regularly exhorted Apple to make cheaper iPhones. With this ‘cheap iPhones by stealth’ strategy as Evans calls it, they just did it without actually having to make any phones. When this particular penny drops with Android OEMs it will be the mobile equivalent of that infamous coffee cup moment in the Usual Suspects…
Devices and Manufacturers
- BuzzFeed on the news that Amazon are to bury the Fire phone, a story that broke on the same day as the Apple launch. Perhaps they hoped no-one would notice.
- Another would-be plucky smartphone contender spun out of the ashes of Nokia’s Meego efforts, Jolla, also seems to be struggling. This week COO and figurehead Marc Dillon stepped aside with The Register suggesting that maybe their best hope lies in being chosen as the “independent” smartphone technology platform of choice by the Russian authorities.
- Nokia themselves meanwhile increasingly seem set to turn to Android as their smartphone platform if persistent rumours are to be believed. Engadget showcased some leaked photos of the C1, alleged to be their first of any sort since the disastrous (for those involved) divestment of their phone division to Microsoft.
- OnePlus confess to making a mess of the launch of the much-anticipated OnePlus Two phone with production and shipment problems leading to huge delays.
- Engadget review the Marshall London phone and find much to like though it seems like something of an acquired taste. Still the launch party sounded like fun:
Marshall dished out shots of JD and put on punk rock performances. At one point, an executive on stage proclaimed the company was going to [verb beginning with F] the competition, in the [orifice beginning with A]. Whether that’s going to catch on like Tim Cook’s “We think you’re going to love it,” I’m not sure. I’ll admit, though, it made me instantly warm to Marshall.
Google and Android
- Reuters on why Google won’t have an easy ride back into China off the back of Google Play:
Google Play is available in China, but reaches only 21 million of an estimated 800 million Chinese mobile users, Warman said. The primary app stores of Internet giants Qihoo, Tencent and Baidu account for two thirds of the market. … These players are unlikely to give away that advantage.
- 10 months after launch, Android Lollipop has just reached 20% adoption:
- Zimperium who first disclosed the infamous Android Stagefright security hole a few weeks ago have now released a Python script that executes the exploit. It isn’t a general purpose exploit since it is limited to one specific device model, a Nexus running Android ICS. Still, it ups the pressure on OEMs to provide fixes. The script itself is full of struct.pack badness.
- TechCrunch preview a few recent high profile Android security issues in a post in which they ask whether they collectively represent “the end of the Android heyday“. In Europe at least, Android as a platform is struggling to make headway against Apple and indeed losing market share according to Kantar.
Apps, Services and Design
- Byte seems to be the hot app of the moment. Built by the creator of Vine, it further blurs the line between applications and the media they create. It’s a little tricky to get your head around but worth sticking with if only to gain a broader understanding of where app interfaces may evolve next:
“THERE’S NO REASON A UI COULDN’T BE A PIECE OF MEDIA.”
- It’s a somewhat old-school term, but advances in human-computer interaction (HCI) have had as crucial a role as Moore’s Law in driving user interface progress into the smartphone era.
- However, changing user interface with every release of your app constitutes “messing with your users” rather than progress:
If you find yourself redesigning your UI with every release, what you are doing isn’t really “design” — it’s just messing with your users.
— Scott Murray (@alignedleft) September 8, 2015
- In equal parts depressing and unsurprising, this post by lava lamp analyses the contents of “paste as a service” proposition Ghostbin and finds it full of dodge:
Services that offer anonymity are great, and they solve some very interesting problems. Even so, anonymity is a tool that is often sought after and used by people without good intentions. While the numbers I’ve crunched above may contain some amount of inaccuracy, the picture remains clear: a significant amount of the pastes shared on Ghostbin have illicit content.
- A great presentation from Pete Cheslock on how the ability of an organisation to leverage the benefit of devops largely depends upon the culture and structure of the organisation. Which helps explain why the Romans would be awesome at devops:
- From the presentation a great quote from Deming should serve to remind those that lead where to look first for problems:
- Apparently WinOps is a thing and having a London Meetup next week to talk about among other things “Chef, Puppet, Ansible, Docker etc.” Wonder if they’ll also mention CRLF-LF gotchas.
- Another great post from Asymco which seeks to understand why Enterprise IT has been so sidelined by the social media analytics and cloud (SMAC) revolution. As usual he nails it:
The problem is that enterprises have a capital use and allocation model which is obsolete. This capital decision process assumes that capital goods are expensive, needing depreciation, and therefore should be regulated, governed and carefully chosen. The processes built for capital goods are extended to ephemera like devices, software and networking.
- Excellent TechCrunch post on why South-East Asia, not the US or Europe, is now the crucible for disruptive mobile business models. It’s a development that has been fuelled by a combination of “no-tail” leapfrogging of Web1.0 paradigms, a mass jump to a smartphone first market and open embrace of eCommerce over ad-based revenue models.
- Underlining the message that the Asian marketplace is becoming more sophisticated, The Telegraph report on luxury brands, China and the ultimate showrooming use case:
“Chinese visitors tend to plan upcoming shopping trips two to three months in advance … and research online price differences between luxury goods bought in their home markets and abroad. This kind of shopping arbitrage has led to the rise of third-party selling agents, known as “Daigou”, which is fuelling the practice of savvy shoppers placing orders abroad.”
- Quartz on a day in the life of a Chinese app addict from Chongqing. It’s a fascinating crazy world dominated by WeChat, JD.com and Taobao.
- Interesting WSJ profile of Wingtech, one of a group of Chinese ODMs that started out in the wild world of Shanzai phone manufacture but are becoming increasingly technically sophisticated, automated and discerning both in terms of the work they take on and the calibre of their customers. Wingtech’s customers include Xiaomi, Huawei, Lenovo and Micromax:
Like most contract manufacturers, Wingtech does a lot more than just manufacture devices. It works with clients to design products, providing lower costs due to its large scale than what its clients could achieve by doing it all in-house.
- No wonder the founder of Hax, Benjamin Joffe, is so bullish about the Chinese hardware accelerator scene:
In the last few years, hardware startup activity in Shenzhen has increased dramatically. It is now comparable to the software innovation system in Silicon Valley.
Wearables and the Internet of Things
- The Raspberry Pi now has an official 7 inch capacitive touch 800×480 display that can be powered through GPIO port. It’s available courtesy of ModMyPi for just over £50. Interestingly there’s also an unofficial 10 inch screen from Kano with integrated battery that might be more suitable for media display projects.
- FastCompany preview the emerging neurohacker scene taking advantage of “the advent of low-cost, easy-to-set-up EEG gear and sophisticated cloud-based AI resources“. The use of transcranial direct current simulation (tDCS) is something that has been covered before in this blog in the context of neurotech startup Thync. What’s new and interesting here is the application of big data analysis across aggregated data sets:
Another technique—and the most intriguing—is bulk analysis of EEG streams, without knowing what the people submitting it are doing or thinking. With enough data and powerful artificial intelligence analysis, the system starts to find chunks of brainwave activity that are common to all people. Essentially, it starts to pick out the individual words and phrases used in the language of the brain.
- Would you let these guys hack your EEG streams?
- In this week’s iWonder, as part of the Make It Digital initiative, the BBC preview 15 key moments in the history of AI including the infamous AI winter made spring by a return to AI’s neural network roots.
- The week BBC Panorama asked Could A Robot Do My Job? The program included an extended interview with Erik Brynjolfsson and Andrew McAfee of MIT, authors of The Second Machine Age. The program suggests 800k jobs have already been lost to automation in the last 15 years in the UK with 10 million more at risk most in the under 30k salary bracket. The rate of progress is following an exponential curve and the presenter makes the key point that humans are really not good at grokking exponential change:
If you are sitting at a desk, driving a taxi or carrying a hod, stop for a moment and ask: could a robot or machine do this job better? … The answer, unfortunately for you, is probably – yes.
- Nick Carr on “The Internet of Too Many Things” and his curious theory that “smartness is a zero sum game“. The more we outsource intelligence, the less of it there is to go about. The idea seems to directly channel Will Self’s Quantity Theory of Insanity:
Each of these products is an example of a defining trend of our networked age: the outsourcing of common sense to gadgetry. A foundational level of human perception and competence is being mechanized through apps and online services. The more mediated our lives become, the more we rely on media to make sense of the world for us. … There is a fixed amount of intelligence in the world, and every time we think we’re creating new intelligence, what we’re actually doing is just redistributing the intelligence that’s already there.
Software Engineering and Management
- This “programmer competency matrix” offers a way to assess where your dev capabilities sit. I’m not sure whether doing this blog represents level 2 or level 3 competency within the provided taxonomy:
- InfoQ interview the author of Go Kit, “an open source microservice toolkit that can be used to facilitate and standardise the creation of Go-based services within the modern enterprise application stack.”
The greatest engineers who have brought us this far in software grew up in an entirely different era of computing — packed inside rooms filled with terminals, carrying mental models of new algorithm-based programs. Newer generations of programmers are born into a more fragmented world.
- TNW on how Live Coding holds the potential to change the way we hire developers by allowing observation of coding at a distance:
- HBR on “why more and more companies are ditching performance ratings“. Basically, they don’t really work and demotivate people. Especially engineers.
- Whatever became of MOOCs? This sharply observed post in The Kernel suggests that they’ve fallen to earth somewhat since the lofty rhetoric of 2011 when they were trumpeted as representing a “revolution in education”. In fact, business model realities have forced a re-evaluation of priorities and MOOCs have essentially morphed into an “Uber for education” providing top-up domain specific training those who already have degrees. It helps contextualise Udacity’s introduction of nanodegrees:
MOOCs were lambasted for having a high dropout rate; the average completion rate still hovers around 15 percent, a level that would be unacceptable for a traditional face-to-face college class. And when the demographics of “successful” MOOC students were scrutinized in one University of Pennsylvania study, it was discovered that 80 percent already had college degrees.
- FirstRound on startup scaling. A kind of exponential growth characteristic of funded outfits that is a curious bedfellow to the their frequently “small agile team” culture origins:
There’s a unique feeling of ambiguity, chaos and stress that comes with doubling or tripling your team every six months. If you don’t manage scaling proactively, you can end up in trouble.
- Crowdsourcing has proved an incredibly successful way of funding project development with success rates in the 90% range. However, according to this Medium post, the biggest challenge the model faces is ensuring sufficient meaningful supply given it is “so foreign to most organizations that few are actually willing to let the crowd model work for them.“
The democratization of the tools of production, combined with the scale of society and increasing amounts of free time, affords more people the opportunity to contribute to projects, in ways large and small, than we could have ever imagined.
- TechCrunch on the need for VC to innovate itself into a shape driven by “investors who prioritize sectors that make a real-world impact increase their bottom lines.” That includes greater geographic, racial and economic diversity. Until that happens, the inconvenient truth behind the Silicon Valley hype will remain in place:
Most technology startups remain relevant only to the best-off in society, leaving out billions of people and trillion-dollar markets.
- TechCrunch on HandUp, a startup that really is trying to make a difference by addressing homelessness in San Francisco and “providing an example for the tech community” in doing so:
Culture and Society
- Stumbled upon a Quora post by Barack Obama explaining why the US-Iran nuclear deal is a good one. Quite aside from the politics it still somehow remains a surprise to see the US President engaging on a social question and answer site.
- Sleep seems to have been getting a lot of coverage lately. This contrarian Quartz post suggests it is a myth that we’re not getting enough of it or as the author somewhat drily puts it:
there is little reason to lose sleep worrying about not sleeping enough.
- This sobering Nautilus post examines the moral gamble implicit in parenthood. The reality is that nobody can know for sure how children will turn out in the long run.
- Staying on the theme of future past, an odd TNW opinion piece which suggests your future self “hates you” because you procrastinate and do things without considering the impact on you in years to come:
When I see someone smoking a cigarette that’s obviously someone who hates his future self so much he doesn’t mind if he gets cancer.