Video Games and Deep Learning

[avatar user=”malm” size=”small” align=”left” link=”file” /]

The view that Machine Learning and Artificial Intelligence are the next big thing for Silicon Valley is mainstream enough to get regular airing in the likes of NYT.  The increasing sophistication of computer gaming algorithms in helping to bring about this about is something that DeepMind founder Demis Hassabis and others have expanded on at length.  The importance of the humble GPU built to improve video game performance in helping to power the revolution is also covered well by Jeff Atwood in a recent post:

Even if you don’t care about video games, they happen to have a profound accidental impact on machine learning improvements. Every time you see a new video card release, don’t think “slightly nicer looking games” think “wow, hash cracking and AI just got 2× faster … again!”

And the subject of this April Fool’s joke announcing a 1000Tflop accelerator board called “The Brain”  because:

It’s a well known fact that all interesting problems can be reduced to dense matrix-matrix multiplication

It seems that scarcely a day passes without another unlikely source wading into the machine learning fray such as this SAS blog offering a breezy attempt to demystify neural nets:

So what’s going on in that hidden layer? Math. Just math. Some pretty funky math, but just math all the same. The default function for neural networks in SAS Enterprise Miner is three nodes/units (out of a range from 1 to 64) of the hyperbolic tangent function, TANH for short, starting with random coefficients (weights) for each unit, which get tweaked as the system “learns” to make better decisions from the training data (H1, H2, H3).

Or this Forbes post which attempts to explain the potential of deep learning suggesting that the ability to actually exploit it fully today is limited to a relative handful of experts:

Right now, deep-learning technology is in its infancy. Maybe 100 people in the world have the true implementation skills to drive it forward. But the possibilities are large. We’re on the brink of an explosion of innovation in this space, both because we’re capturing so much more data about everything in the world, and because we’ve got so much more computing power to process and analyze it all.

As Machine and Deep Learning technology matures and travels up the S-curve, however, it is increasingly likely to be provided as a commodity service like cloud computing is today.  This is the prime territory that Google are attempting to move into with their Google Cloud Platform (GCP) Machine Learning play.  When Deep Learning is available on tap, it will transform the work landscape profoundly, a fact not lost on developers, many of whom are already worried about the impact on their own employment:

The thought of obsolescence due to AI was also more threatening than becoming old without a pension, being stifled at work by bad management, or by seeing their skills and tools become irrelevant. While the developers who worried about AI were found across industries and platforms, one strong correlation was that they were more likely to self-identify with being loners rather than team players.

With everyone from developers to even CEOs susceptible to technological unemployment, the impact of automation on jobs and indeed on the whole capitalist model is increasingly being raised as a subject of discussion in technology blogs.  TechInsider pointed out two key trends suggesting why.  Firstly clear evidence that technology has been hollowing out manufacturing over several decades:

Secondly a startling data point from the CEO of NVidia, purveyors of the sort of GPU hardware that is powering the deep learning revolution, makes clear the sheer scale of business interest in deploying it:

Two years ago we were talking to 100 companies interested in using deep learning. This year we’re supporting 3,500 [companies]

Among those in the stampede into the promised land are nation states doubtless aware of the potential it entails to track terrorists and other dissidents.  The Chinese authorities are even making a point of emphasising they are intent on building a Minority Report style “pre-crime” system:

The Chinese government wants to know about everything: every text a person sends, every extra stop they make on the way home. … By publicly announcing their intention to build an intelligence network that can predict crimes, China just took a step closer to all the thought-policing dystopian nightmare scenarios we’ve always worried about as members of a modern society. And they want people to know it.

George Orwell could not have anticipated the specific technological advances in machine learning that enabled his dark totalitarian vision to happen but even so, many of the ideas he advanced in 1984 seem to be finding their realisation in approach to Singularity as highlighted before. Ironically enough in part powered by our appetite for ever more life-like video games.  Something to think about the next time you find yourself or someone else playing one.

Apps and Services

• The debacle over Tay, the ‘racist’ AI bot released and then hastily withdrawn by Microsoft could easily have been avoided through better preparation before launch.

what does it mean to build bots ethically?  … The basic takeaway is that botmakers should be thinking through the full range of possible outputs, and all the ways others can misuse their creations.

• Good Wired profile of Netflix and their ‘crazy plan to conquer the world’.

• A BuzzFeed on why LinkedIn is “simply the worst” highlights some of its foibles.

• Prolific tweeter and creator of ZeroMQ, Pieter Hintjens, on “Why I’ve quit Twitter“:

We’ve all seen the divisive arguments over race and gender. These aren’t just your usual Internet arguments. These are becoming a civil war and I’m stepping out of it.

• Chine Uncensored ask whether Facebook’s founder is Zucking up to the Chinese:

Cloud and Open Source

• “HTTPS is hard” offers a really good insight into why it is hard to implement a supposedly ‘simple’ technology requirement change across a large legacy organisation.  In this case the company in question was Yell and the project was the transition to serving its content over HTTPS rather than HTTP:

Altogether, getting the HTTPS project from ideation to go-live took 7 months,and involved over 10 internal departments and five external agencies, visable in the diagram below:

• Hell just froze over.  Windows 10 will integrate Bash shell support in a move that would have been utterly inconceivable 15 years ago when Microsoft were utterly dominant and Linux a pinprick of annoyance:

• ArsTechnica provided a little more detail on how it will work using Microsoft’s “WSL”:

the company has developed some Windows kernel components (lxcore.sys, lxss.sys, presumably standing for “Linux core” and “Linux subsystem,” respectively) that support the major Linux kernel APIs. These components are not GPLed and do not appear to contain Linux code themselves; instead, they implement the Linux kernel API using the native Windows NT API that the Windows kernel provides. Microsoft is calling this the “Windows Subsystem for Linux” (WSL).

• InfoWorld on Google’s attempts to take on AWS and Azure with GCP to make public cloud a three horse race:

AWS is way out ahead, and Microsoft is next in line, but way behind. Google seems to be in last place, but is increasing its efforts.

Security

• Quartz survey suggesting 20% of employees would sell their work passwords, some for less than $100.

• Android Factory Reset Protection is a good example of an important security feature with unintended side-effects. Designed to make it impossible for opportunistic phone thieves to factory reset stolen Android devices by tying the action to your Google account, it has ended up catching out many others who passed on phones legitimately forgetting to first switch it off.  This post essentially emphasises why it is really important to disable it if you’re transferring your phone to someone else .

Mobile and Devices

• A good review of the iPhone SE compares it with the iPhone5s and boils down to one fundamental conclusion:

It’s a blazing fast smartphone for those who don’t think bigger is better.

• With iPhone sales apparently past a point of inflection, Christian Lindholm of Korulab among others explain why “the next big thing in phones might not be a phone“.

Many firms are experimenting with new ways to help consumers interact with the wider world through touch, sight and sound.  These include voice-activated personal assistant devices dangling from “smart jewelry” necklaces with tiny embedded microphones or tiny earpieces that get things done for us based on our verbal commands.

• PSFK on another growing trend in mobile phone space, namely a reversion to “dumb” phones that eschew nearly all distracting smartphone features in favour of simplicity. One such example being the Swiss manufacturer Punkt with their somewhat pompously-named MP01:

The phone’s primary function is also to receive and make calls and text messages (limited to a 2G network), but it also has an alarm clock and calendar function. It costs $295.

Hardware and IoT

• Good and approachable explanation of how shift registers work:

• Thoughts on designing a home network for hostile devices.  It seems having a lot of knowledge of networking is a prerequisite.

• Talking of which, here’s a guide to setting up integrated Bluetooth on the new Raspberry Pi 3:

Python

• EuroPython 2016 agenda has come out and reflecting wider developer concerns and interests has several interesting looking talks related to machine learning including one entitled deep learning with Python tensorflow and another on building your own AI.

Science

• Can an intelligent living creature be as big as a galaxy?   “Probably not” and this post explains why:

Interesting thoughts require not only a complex brain, but also sufficient time for formulation. The speed of neural transmissions is about 300 kilometers per hour, implying that the signal crossing time in a human brain is about 1 millisecond. A human lifetime, then, comprises 2 trillion message-crossing times. … If both our brains and our neurons were 10 times bigger, and our lifespans and neural signaling speeds were unchanged, we’d have 10 times fewer thoughts during our lifetimes.

Autonomous Vehicles

• The week when Tesla 3 launched started with Quartz laying out the epoch-defining importance of a success for affordable electric vehicles:

The stakes are the highest ever for Musk. If motorists buy the Model 3 in the hundreds of thousands, he will have delivered on his vow to make an electric for the general public. The consequences of all this turning out well could be considerable profit for Musk and his investors, not to mention a new upheaval in geopolitics.

• When the Model 3 was finally announced, its release date was pegged to 2017 with a starting price of $35,000.  Crucially within a couple of days, Elon Musk was able to announce a remarkable 200k pre-orders had been made each involving a downpayment of $1000.  It’s a remarkable triumph and vindication of Musk’s approach, in part due to extraordinary technology but also partly down to extraordinary marketing.  40 years on from the birth of Apple, it seems his natural successor is creating his own “reality distortion field”, one that effectively turns every Tesla customer into a venture capitalist.  Jobs probably wouldn’t have done it this way but would likely have approved the hype creation.  What remains now for Musk is to deliver:

Thought it would slow way down today, but Model 3 order count is now at 198k. Recommend ordering soon, as the wait time is growing rapidly.

— Elon Musk (@elonmusk) April 1, 2016

Startups

• Elon Musk is rightly lauded but in all the excitement it is worth remembering that he is an immigrant like a very large proportion of the founders of US unicorn startups.  This National Foundation for American Policy (NFAP) report provides a detailed breakdown:

Immigrants have started more than half (44 of 87) of America’s startup companies valued at $1 billion dollars or more and are key members of management or product development teams in over 70 percent (62 of 87) of these companies. The research finds that among the billion dollar startup companies, immigrant founders have created an average of approximately 760 jobs per company in the United States. The collective value of the 44 immigrant-founded companies is $168 billion, which is close to half the value of the stock markets of Russia or Mexico.

Society and Politics

• Andy Grove, the CEO who guided Intel to world dominance passed away recently.  He was famed for his quote that “only the paranoid survive” and wrote a “Warning to Silicon Valley” essay in 2010 urging “job-centric” economics and politics, involving an all-out commitment to American manufacturing. Interestingly, in light of the immigrant data, Grove was Hungarian-born.

• Meanwhile the reality of crappy customer support gigs in the “new economy” is laid bare in this brutal assessment of “dead end work in the boondocks”.

• Relatedly Salon explain how “shameless financical strip mining” has “killed the middle class” in the US irrespective of the election results in an interview with Les Leopold.  He is the author of what looks like an essential read just from the cover picture:

• At least the Americans have religion to fall back on unlike it seems increasingly the Europeans.  This map shows the distribution of belief in God across the two continents:

Big difference. Belief in God. North America vs Europe
Via @amazingmap https://t.co/79tBbY2oet pic.twitter.com/2lCVnJQFhc

— Paul Kirby (@paul1kirby) March 26, 2016

Trump

• Meanwhile “The Donald” only has one core belief according to this devastating Slate analysis and that is misogyny.   This week he demonstrated it in spades with disgraceful comments about ‘punishment’ for women who undergo abortion that even he had to row back on in haste.

• Quartz also had a look at the bizarre ‘us vs. them’ psychology of Trumpism and its supporters:

• It’s going to be difficult for him to keep it all in check over he coming months.  And Quartz are reporting that America may finally be getting bored of his antics anyway and suggest the precedent of his ratings on the US Apprentice which started off spectacularly well but fell away over time:

Political analysts have long predicted that Trump’s offensive rhetoric would eventually make him politically radioactive, but that hasn’t happened.  …  He’s still the odds-on favorite to emerge with the Republican nomination this summer, but when it comes to durable appeal, history suggests there are diminishing returns. As his negatives rise and his offensiveness becomes trite, maybe voters will just change the channel.