When I had a drink with Andrej Karpathy a couple of weeks ago, we got to talking about where we thought machine learning was going over the next few years. Andrej threw out the phrase “Software 2.0”, and I was instantly jealous because it captured the process I see happening every day across hundreds of projects. I held my tongue until he got his blog post out there, but now I want to expand my thoughts on this too.
The pattern is that there’s an existing software project doing data processing using explicit programming logic, and the team charged with maintaining it find they can replace it with a deep-learning-based solution. I can only point to examples within Alphabet that we’ve made public, like upgrading search ranking, data center energy usage, language translation, and solving Go, but these aren’t rare exceptions internally. What I see is that almost any data processing system with non-trivial logic can be improved significantly by applying modern machine learning.
This might sound less than dramatic when put in those terms, but it’s a radical change in how we build software. Instead of writing and maintaining intricate, layered tangles of logic, the developer has to become a teacher, a curator of training data and an analyst of results. This is very, very different than the programming I was taught in school, but what gets me most excited is that it should be far more accessible than traditional coding, once the tooling catches up.
The essence of the process is providing a lot of examples of inputs, and what you expect for the outputs. This doesn’t require the same technical skills as traditional programming, but it does need a deep knowledge of the problem domain. That means motivated users of the software will be able to play much more of a direct role in building it than has ever been possible. In essence, the users are writing their own user stories and feeding them into the machinery to build what they want.
Andrej focuses on areas like audio and speech recognition in his post, but I’m actually arguing that there will be an impact across many more domains. The classic “Machine Learning: The High-Interest Credit Card of Technical Debt” identifies a very common pattern where machine learning systems become embedded in deep stacks of software. What I’m seeing is that the problem is increasingly solved by replacing the whole stack with a deep learning model! Taking the analogy to breaking point, this is like consolidating all your debts into a single loan with lower payments. A single model is far easier to improve than a set of deeply interconnected modules, and the maintenance becomes far easier. For many large systems there’s no one person who can claim to understand what they’re actually doing anyway, so there’s no real loss in debuggability or control.
I know this will all sound like more deep learning hype, and if I wasn’t in the position of seeing the process happening every day I’d find it hard to swallow too, but this is real. Bill Gates is supposed to have said “Most people overestimate what they can do in one year and underestimate what they can do in ten years“, and this is how I feel about the replacement of traditional software with deep learning. There will be a long ramp-up as knowledge diffuses through the developer community, but in ten years I predict most software jobs won’t involve programming. As Andrej memorably puts it, “[deep learning] is better than you”!
Pete Warden's blog 
To me it seems like a case of “if all you have is a hammer, everything looks like a nail”. Of course, DL will be increasingly more important at companies like Google, Tesla and other “companies at scale”. But most lines of code are being written for cases, where one is not in a position to be “a teacher”. These software projects are very small and very specific. How will DL work there?
How much ML consulting have you done in the world of SMEs who want to do ML? Does this still hold true there?
It’s the combination of traditional programming skills with machine learning skills that will have the most impact in the short to mid term. Once ML tools and meta-tools mature this will shift more towards ML.
I spent over 20 years in the semiconductor industry. The EDA tools used by designers seems like a disruption opportunity to me – very high per-seat pricing, hard to use, convoluted & constantly changing tool chains, … I could go on.
I think within 5-10 years, kids will be training their ML pets at age 5, and it will continue throughout one’s education. Programmers will continue to operate in dark corners.
Reblogged this on Bipin's Blogs.
“For many large systems thereās no one person who can claim to understand what theyāre actually doing anyway, so thereās no real loss in debuggability or control”. Is this Software Engineering surrender?
This article gets across the big points well. Example, this part on the need for domain expertise is spot on, “The essence of the process is providing a lot of examples of inputs, and what you expect for the outputs. This doesnāt require the same technical skills as traditional programming, but it does need a deep knowledge of the problem domain.”
Pingback: Are deep neural nets "Software 2.0"? - Michael S. Chimenti's Bioinformatics Blog
Great post! I have been thinking a lot about the end of coding as we know it. Much like hardware- we once had to design functions part by part, now they are a ‘black box’ used without need to know the internal details at all.
I am sure AI tools will get us to that point soon.
I do wonder about debugging as the process will become very removed and not understandable by humans. Diagnosing problems could become like a medical doctor trying to fix a complex organ whose internal operations are not clearly understood.
It will be interesting indeed. Kids should combine human education courses along with coding. Whatever code language they learn will be obsolete- they will deal with the machine on a much different level.
I, for one, welcome our new computer overlords! Thanks for a good read!
Pingback: Deep Learning Is Eating Software – JeffPerales.com
Pingback: Machine learning is changing the way we build software – chloedo.es
Pingback: Newsletter 37 ā Software 2.0 | import digest
Here’s my thought process about machine learning:
Q. Do I have a large database of interesting examples to train on?
A. Not for the problem I’m trying to solve.
Q. Can I generate it somehow?
A. We won’t even begin to collect it until after we go live.
Okay, never mind then.
It sounds great for people who do have the data, though. Maybe in 2.0.
Very interesting to discuss about the future of Deep Learning and AI. Programming logic on ML is very different with traditional programming. We must concern about this topic because in this year robotic industry had a very significant progress and prove that they can build a humanoid robotic (Sophia) that use ML and AI. I think that in the next years (maybe 10-20 years) this world just like Terminator Film.
Pingback: Deep Learning estĆ” comiendo software – El Neurotransmisor
Pingback: Links for November 7th through December 28th
Yes, it is right, but sometimes software eats too š
Pingback: How to Compile for ARM M-series Chips from the Command Line « Pete Warden's blog
Pingback: 6 Ways Machine Learning Transforms Software Development - Metamaven
Pingback: 6 Ways AI Transforms How We Develop Software - Mariya Yao
I doubt your whole premise. These things are cyclical. The more folks realize that they canāt reproduce the same result consistently, even with the exact same data set, the faster their enthusiasm fades. Itās like all these research unicorns are farting bad numbers into poorly designed tools then calling any result that emerges an AI. The real world is messy. Data about it is inherently sensitive to initial states. This becomes more like guessing than prediction. Itās not ml, or AI, itās delusional fairy gas.
Pingback: reconstitute the world « Bethany Nowviskie
Pingback: What machine learning means for software development – f1tym1
Pingback: What machine learning means for software development | Tesco Inc.
Pingback: What machine learning means for software development - O'Reilly Media
Pingback: What machine learning means for software development – A.I WARE
Pingback: 6 New Trends in Software Development
Pingback: åƹäŗč½Æ件ē åččØęŗåØå¦ä¹ ęå³ēä»ä¹
Pingback: Industry experts weigh in on the adoption of AI and ML in software development | Fingent Blog
Pingback: 4 Ways AI (Artificial Intelligence) Improves Software Development