The Human in the Machine

[avatar user=”malm” size=”small” align=”left” link=”file” /]

The role of Human-Assisted AI (or HAAI) in supporting human and machine intelligence for service delivery is a topic that has been covered at length already in this blog in 2016.  A recent ComputerWorld article refers to it as “human in the loop computing”:

If the confidence score is below a certain value, it sends the data to a human annotator to make a judgment. That new human judgment is used both for the business process and is fed back into the machine learning algorithm to make it smarter. In other words, when the machine isn’t sure what the answer is, it relies on a human, then adds that human judgment to its model.

People rather than algorithms remain the often unhappy grunt force behind pretty much all “AI bot” propositions today as this excellent recent Bloomberg post revealingly entitled “the humans hiding behind the chatbots“ points out:

A handful of companies employ humans pretending to be robots pretending to be humans. In the past two years, companies offering do-anything concierges (Magic, Facebook’s M, GoButler); shopping assistants (Operator, Mezi); and e-mail schedulers (X.ai, Clara) have sprung up. The goal for most of these businesses is to require as few humans as possible. People are expensive. They don’t scale. They need health insurance. But for now, the companies are largely powered by people, clicking behind the curtain and making it look like magic.

The “Wizard of Oz” pattern is an approach that was pioneered in the UK in the Noughties by text question and answer service AQA and voice to text service provider SpinVox.  AQA were relatively transparent about the humans in their SMS loop.  SpinVox, however, were notoriously more secretive about their ‘advanced’ AI technology (called ‘D2’) until they were eventually unmasked.  AI even back then was a big draw and their story provides a cautionary tale for today’s HAAI aspirants:

Wind on a few years to the present and the “app as assistant” paradigm has re-emerged as the next big thing.  Bot hype is everywhere with VCs amongst the most bullish in their pronouncements.  There’s no doubt the AI this time around is far more sophisticated and powerful:

Products like Fin/Operator/M are at least partially powered by real humans. Other important ingredients are NLP to process text inputs,CV and deep learning improvements in audio-to-text to process inputs like audio/photos/video, and more. Its so much more than text and AI.

Even so, it’s not clear we really know understand how to leverage techniques like Deep Learning to build Conversational UIs yet:

Truly ‘conversational’ software can manage a complex, evolving state. It can use knowledge provided a priori by the developer, and can gain new knowledge from users. That’s a difficult problem to solve.

It’s unlikely the typical chatbot startup can make much progress in this territory without serious PhD level research chops on board in niche techniques most of us have never heard of, like DMN:

Or indeed, if they get there, that IM chat-based messaging is the right interface anyway.  Dan Grover, a Tencent employee and WeChat product manager based in Guangzhou China, has published an essential read on how the WeChat service interface owes a lot of its success to integrated calls to action in the form of lists and other buttons as well as offering a unified platform interface to features such as payments.  It’s an approach being adopted by Telegram bots and within Facebook to work around mobile platform shortcomings:

It’s worth reflecting on these two considerations by way of backdrop next time you read yet more hype of Facebook’s M virtual assistant that “lives inside of Messenger”:

• The degree of dependence on an underlying Mechanical Turk model.  To what extent is it humans all the way down?
• The possibility that conversational UI isn’t the right paradigm for a rich service interface anyway.

Messenger and M integration is clearly a huge deal to Facebook. Reading between the lines, however, it is still very much a work in progress with seamless mobile shopping a particular area of focus. Early feedback from Gizmodo on the handful of M chatbots themselves is perhaps unsurprisingly disappointing highlighting a number of characteristic limitations such as  global amnesia and long uncomfortable pauses.  It doesn’t make for a very human experience.

Poncho is basically looking for one particular phrase, and that’s “Do I need [insert item]?” As you can see in the screenshots above, referring to my past chats or previous answers doesn’t work (you know, like a normal conversation). Every chat with Poncho exists in a vacuum and it forces you into a robotic back and forth, no matter how Poncho disguises it—be it with funny GIFs or pop culture references.

Google and Android

• The escalation of the EU’s anti-bundling crackdown on Android represents a ‘dangerous’ new front against Google in Europe and further exposes the fault lines between the EU and US.  Politico goes as far as to claim it is “Google’s worst nightmare“.  The timing is certainly bad given the momentum behind Android and Google’s attempts to use it to break into the IoT space.  It harks back to the bad old days of the Microsoft EU Antitrust case over a decade ago.  In fact, the two companies are engaged in an elaborate industry mirror play:

Android has become the Windows of the mobile world, serving as the ubiquitous operating system of choice, and Google’s software licensing terms are leading device manufacturers to bundle and promote its apps ahead of others. Another commonality is that Google was one of the behind-the-scenes agitators during Microsoft’s IE tribulations, and Microsoft returned the favor in 2013 when it petitioned the EC to look into Android app bundling.

• It’s a development that will doubtless reverberate around Washington.  The Intercept highlighted the sheer extent of interaction between Google and the White House over the last 7 years with effectively an open door between the two and the suggestion that Barack Obama has presided over “The Android Administration”.

• Android M (Marshmallow) market share doubled last month and now stands at 4.6%.

• Google published in March a collection of “principles of mobile app design” covering their view on best practice across most typical app touchpoints.

Big Companies, Big Mistakes

• With Google re-enacting Microsoft history, it’s worth reflecting on what a still Microsoft-dominated world of 2007 was like. Steve Ballmer laughing at the iPhone v1.0 “the most expensive phone in the world” should be required viewing for industry executives who think they know the business better than their customers. This is literally what disruptive innovation from above looks like for incumbents:

• Microsoft nearly a decade on is a much chastened company with Ballmer himself long gone.   Current CEO Satya Nadella has certainly helped turn the ailing ship around with a focus on Azure cloud technology. However, the extent of the titanic struggle the company faces to compete with new challengers is highlighted in this post.

• They weren’t the only company wrong-footed by Apple.  Intel also made a huge mistake 10 years ago:

Intel turned down an opportunity to provide the processor for the iPhone, believing that Apple was unlikely to sell enough of them to justify the development costs.

• And now like Nokia before them, 12,000 workers being laid off are paying the price:

Intel will lay off 11% of its global workforce, up to 12,000 employees, a painful downsizing aimed at accelerating its shift away from the waning PC market to one more focused on cloud computing and connected devices.

• Apple themselves seem to have fallen into an internal culture clash not dissimilar to that which afflicted Nokia in it’s pomp when different teams expended their energies on each other rather than external threats.  Presumably Apple management are far too wise to history to let the sort of wrangling apparently happening within their cloud computing division from dragging on much longer.

Artificial Intelligence

• Tensorflow playground is an awesome visualisation of different neural nets in action during training:

• Slate article outlining “the philosophical argument against AI killing us all” which rests in essence on the notion of a morality being an emergent property of any sufficiently complex ‘computronium’ be it organic or inorganic:

“the escalator of reason leads societies to greater benevolence regardless of species origin. A.I.s will have to step on the escalator of reason just like humans have, because they will need to bargain for goods in a human-dominated economy and they will face human resistance to bad behavior.”

• The FT have run a ‘special’ on AI and robotics.

• A Chinese ‘tuhao’ goes shopping with eight robot maids who have the job of carrying his purchases.  The term is used to describe ostentatious shows of wealth by the newly minted and has a distinct degree of derision attached to it:

Security

• Author of ssh chat asks “Why aren’t we using ssh for everything?“

ssh http://t.co/E7Ilc0B0BC pic.twitter.com/CqYBR1WYO4

— Andrey 🦃 Petrov (@shazow) December 13, 2014

• Good CESG article exposing the spurious rigour involved in enforcing regular password expiry:

The problem is that this doesn’t take into account the inconvenience to users  – the ‘usability costs’ – of forcing users to frequently change their passwords. The majority of password policies force us to use passwords that we find hard to remember. Our passwords have to be as long as possible and as ‘random’ as possible. And while we can manage this for a handful of passwords, we can’t do this for the dozens of passwords we now use in our online lives.

The Internet of Things

• On IoT consortia:

In consumer #IoT, are there actually more interoperability consortia than things worth interoperating?

— Ari Jaaksi (@jaaksi) April 14, 2016

• Another day, another dodgy IoT use case.  This is the sort of thing that gives IoT a bad press amongst ordinary consumers:

the future, finally delivered pic.twitter.com/0b7Vz3BUGm

— Internet of Shit (@internetofshit) April 23, 2016

Manufacturers and Devices

• Amazing MIT shapeshifting display:

• InFocus is Foxconn’s consumer smartphone brand.  Here’s a review of the sub-£80 (at least in some places online) InFocus 4G model the M560.  It’s hard to compete with these specs at that price:

Software and Programming

• Notes on how to do multithreading in Python with BeautifulSoup properly.

• Jaspy is “Jaspy is a Python VM written entirely from scratch in JavaScript with some unique features“.  It allows you to debug client side web code interacting with the DOM inside of PyCharm:

• I managed to add a little bit more to my Swift cloud notebook.

On the art of Fixing Things

• This week’s episode of The Reassembler has ex-Top Gear host James May reconstructing a completely disassembled electric guitar over six hours.  It’s as much fun listening to him talking about the history of different screwdriver types as it is following the instrument rebuilding.

• The Restart Project just released a short video about another large scale problem facing humanity – electronic waste.  They too encourage a reuse and repair philosophy and run workshops to help those .   A key learning is to avoid panic and overreaction when Things Fall Apart and instead learn to stay calm and think through the issues.  It’s an approach that serves equally well with software.

Science

• Future solar panels “will generate energy from raindrops” too courtesy of a graphene coating.  Generating energy from rain must surely be a priority for the UK.

• The potential for DNA to serve as a substantial storage medium:

DNA can theoretically store one exabyte of data in a volume of a grain of sand. That’s roughly eight magnitudes denser than that of tape, and roughly equivalent to 200 million DVDs. … What’s more, DNA is extremely stable. The average lifespan for rotating disks and optical storage systems is at most a few decades.  In contrast, DNA has the potential to reliably store data for centuries without significant decay.

• MIT Technology Review examine the curious case of the ‘impossible’ EmDrive thruster and the suggestion it’s apparent operation could be explained by quantisation of inertia at low acceleration.

• Aurora Borealis and Australis viewed from the ISS:

Culture and Society

• “The Secret Rules of the Internet” is a long Verge post examining the emergent history of content moderation and how it relates to free speech.

• Technology is driving us inexorably towards a post-capitalist era but according to this post in The Information, it’s not clear that what comes next will be an improvement:

instead of outgrowing traditional capitalism, technology is going to erode the base of capitalism in the next few decades. Rather than building on top of a strong capitalist base, we will be dismantling capitalism as we build something new.

• “It’s not worth me having a long life lightbulb” and other attitudes towards death amongst the very old as revealed in a Cambridge University cohort study of over-95’s.  The imminence of death is a mainstay with many reporting they take each day at a time as it comes.

• The mess that is the Middle East:

I can't stop thinking about this Philip Gordon quote https://t.co/KbiNthETIZ pic.twitter.com/JVup6hJv24

— David A. Graham (@GrahamDavidA) April 15, 2016

Work and Play

• HBR have just published an article entitled “Embracing Agile” which represents something of a watershed.  Although Agile development is an approach that has long been championed in IT and software, it has arguably never been considered relevant for reinventing the wider business environment. Key ingredients in the mix they identify include a structured process built around multidisciplinary self-governing teams, stack-ranking tasks for sprints, a clear definition of done and above all, full endorsement and support from the very top.

• NYT on why speed reading doesn’t work:

it’s extremely unlikely you can greatly improve your reading speed without missing out on a lot of meaning. Reading is about language comprehension, not visual ability.

• Elon Musk is literally a man from the future in the sense that his vehicles are able through software update to get better over time after sales unlike any other vehicle before them.   Even he still only needs a single page CV though according to online résumé-writing firm Novorésumé in the ultimate proof that less is more:

• Quartz on why Generation X (current age 46-51) have been particularly badly affected by recession – in real dollar terms their salaries have been flatlined since 2008 as highlighted in the graph below:

Hardly anyone has gotten a significant raise since the recession. Gen Xers are worth singling out because they’re in their peak earning years. They’ve now missed out on what should be the biggest raises of their careers.

• Long fascinating NYT article on ‘the Minecraft Generation’ of Centennials for whom the game is a core part of daily life:

Since its release seven years ago, Minecraft has become a global sensation, captivating a generation of children. There are over 100 million registered players, and it’s now the third-best-­selling video game in history, after Tetris and Wii Sports.

• It is at heart ‘an incredibly complex game’ that encourages computational thinking and fluency with the command line that are both key to software engineering later in life.  The author relates this illustrative anecdote:

One day last fall, I visited Gus, a seventh ­grader in Brooklyn.  … I watched as he typed a command to endow himself with a better weapon: “/give AdventureNerd bow 1 0 {Unbreakable:1,ench:[{id:51,lvl:1}],display:{Name:“Destiny”}}.” What the command did was give a bow-­and-­arrow weapon to AdventureNerd, Gus’s avatar; make the bow unbreakable; endow it with magic; and name the weapon Destiny, displayed in a tag floating over the weapon.

• Here’s something equally inscrutable.  Steve Davis, forever associated with anyone of a certain age with almost total dominance of the world of snooker in the 80’s, is reinventing himself as a techno DJ:

Prince

• The news that another 80’s rock superstar had died took a while to sink in and then generated an inevitable outpouring of shock and sadness.   He left behind an incredible back catalogue and by some accounts 26 albums worth of unreleased material.  One suspects his well-documented troubles with Warner Brothers will rumble on after his death with the amount of money they could make from him now he’s dead.  And so the dead live on without recourse to cryogenics.

• Musical highlights abound of course with several key live performances available in various degrees of graininess on YouTube if not as Spotify tracks.  His extraordinary strange performance of Radiohead’s Creep at Coachella and celebrated rendition of Purple Rain at the SuperBowl help underline what an amazing live performer he was in an era when genuine musical ability really mattered.  The virtuoso guitar showmanship comes across well here:

https://youtu.be/F8BMm6Jn6oU

• If you have any remaining doubt as to his rock credentials, do yourself a favour and go 3:28 seconds into the ‘supergroup’ rendition of ‘While My Guitar Gently Weeps’ below performed in celebration of the late George Harrison 12 years ago in 2004. He totally owns the next 3 minutes from there to his Hendrix-like flourish at the end.  As the man put it in Around the World in a Day (1985):

“I can be a weirder fey psychedelic badass mushroom than you, Paul McCartney.”

• There’s a distinct irony here.  Prince doubtless respected the Beatles and Harrison in particular as a fellow guitar genius but he also saw them as representatives of a white rock scene that routinely marginalizes black rock musicians and puts them in a separate category when in fact it’s their scene that has been appropriated by the majority:

When rock music is defined by and limited to white people, only white people get to fit in the critically acclaimed pantheon of rock geniuses. Black performers are made to seem an exception in the genre they invented. … Prince came from a long line of eccentric rock geniuses, and part of how he honored them was by forcing even the most obtuse listener to understand that rock was, and always has been, black music.