Deep Learning is Eating Software


Photo by John Watson

When I had a drink with Andrej Karpathy a couple of weeks ago, we got to talking about where we thought machine learning was going over the next few years. Andrej threw out the phrase “Software 2.0”, and I was instantly jealous because it captured the process I see happening every day across hundreds of projects. I held my tongue until he got his blog post out there, but now I want to expand my thoughts on this too.

The pattern is that there’s an existing software project doing data processing using explicit programming logic, and the team charged with maintaining it find they can replace it with a deep-learning-based solution. I can only point to examples within Alphabet that we’ve made public, like upgrading search ranking, data center energy usage, language translation, and solving Go, but these aren’t rare exceptions internally. What I see is that almost any data processing system with non-trivial logic can be improved significantly by applying modern machine learning.

This might sound less than dramatic when put in those terms, but it’s a radical change in how we build software. Instead of writing and maintaining intricate, layered tangles of logic, the developer has to become a teacher, a curator of training data and an analyst of results. This is very, very different than the programming I was taught in school, but what gets me most excited is that it should be far more accessible than traditional coding, once the tooling catches up.

The essence of the process is providing a lot of examples of inputs, and what you expect for the outputs. This doesn’t require the same technical skills as traditional programming, but it does need a deep knowledge of the problem domain. That means motivated users of the software will be able to play much more of a direct role in building it than has ever been possible. In essence, the users are writing their own user stories and feeding them into the machinery to build what they want.

Andrej focuses on areas like audio and speech recognition in his post, but I’m actually arguing that there will be an impact across many more domains. The classic “Machine Learning: The High-Interest Credit Card of Technical Debt” identifies a very common pattern where machine learning systems become embedded in deep stacks of software. What I’m seeing is that the problem is increasingly solved by replacing the whole stack with a deep learning model! Taking the analogy to breaking point, this is like consolidating all your debts into a single loan with lower payments. A single model is far easier to improve than a set of deeply interconnected modules, and the maintenance becomes far easier. For many large systems there’s no one person who can claim to understand what they’re actually doing anyway, so there’s no real loss in debuggability or control.

I know this will all sound like more deep learning hype, and if I wasn’t in the position of seeing the process happening every day I’d find it hard to swallow too, but this is real. Bill Gates is supposed to have said “Most people overestimate what they can do in one year and underestimate what they can do in ten years“, and this is how I feel about the replacement of traditional software with deep learning. There will be a long ramp-up as knowledge diffuses through the developer community, but in ten years I predict most software jobs won’t involve programming. As Andrej memorably puts it, “[deep learning] is better than you”!

31 responses

  1. To me it seems like a case of “if all you have is a hammer, everything looks like a nail”. Of course, DL will be increasingly more important at companies like Google, Tesla and other “companies at scale”. But most lines of code are being written for cases, where one is not in a position to be “a teacher”. These software projects are very small and very specific. How will DL work there?

  2. It’s the combination of traditional programming skills with machine learning skills that will have the most impact in the short to mid term. Once ML tools and meta-tools mature this will shift more towards ML.

  3. I spent over 20 years in the semiconductor industry. The EDA tools used by designers seems like a disruption opportunity to me – very high per-seat pricing, hard to use, convoluted & constantly changing tool chains, … I could go on.

  4. I think within 5-10 years, kids will be training their ML pets at age 5, and it will continue throughout one’s education. Programmers will continue to operate in dark corners.

  5. “For many large systems there’s no one person who can claim to understand what they’re actually doing anyway, so there’s no real loss in debuggability or control”. Is this Software Engineering surrender?

  6. This article gets across the big points well. Example, this part on the need for domain expertise is spot on, “The essence of the process is providing a lot of examples of inputs, and what you expect for the outputs. This doesn’t require the same technical skills as traditional programming, but it does need a deep knowledge of the problem domain.”

  7. Pingback: Are deep neural nets "Software 2.0"? - Michael S. Chimenti's Bioinformatics Blog

  8. Great post! I have been thinking a lot about the end of coding as we know it. Much like hardware- we once had to design functions part by part, now they are a ‘black box’ used without need to know the internal details at all.
    I am sure AI tools will get us to that point soon.

    I do wonder about debugging as the process will become very removed and not understandable by humans. Diagnosing problems could become like a medical doctor trying to fix a complex organ whose internal operations are not clearly understood.

    It will be interesting indeed. Kids should combine human education courses along with coding. Whatever code language they learn will be obsolete- they will deal with the machine on a much different level.

    I, for one, welcome our new computer overlords! Thanks for a good read!

  9. Pingback: Deep Learning Is Eating Software –

  10. Pingback: Machine learning is changing the way we build software –

  11. Pingback: Newsletter 37 – Software 2.0 | import digest

  12. Here’s my thought process about machine learning:

    Q. Do I have a large database of interesting examples to train on?
    A. Not for the problem I’m trying to solve.
    Q. Can I generate it somehow?
    A. We won’t even begin to collect it until after we go live.

    Okay, never mind then.

    It sounds great for people who do have the data, though. Maybe in 2.0.

  13. Very interesting to discuss about the future of Deep Learning and AI. Programming logic on ML is very different with traditional programming. We must concern about this topic because in this year robotic industry had a very significant progress and prove that they can build a humanoid robotic (Sophia) that use ML and AI. I think that in the next years (maybe 10-20 years) this world just like Terminator Film.

  14. Pingback: Deep Learning está comiendo software – El Neurotransmisor

  15. Pingback: Links for November 7th through December 28th

  16. Pingback: How to Compile for ARM M-series Chips from the Command Line « Pete Warden's blog

  17. Pingback: 6 Ways Machine Learning Transforms Software Development - Metamaven

  18. Pingback: 6 Ways AI Transforms How We Develop Software - Mariya Yao

  19. I doubt your whole premise. These things are cyclical. The more folks realize that they can’t reproduce the same result consistently, even with the exact same data set, the faster their enthusiasm fades. It’s like all these research unicorns are farting bad numbers into poorly designed tools then calling any result that emerges an AI. The real world is messy. Data about it is inherently sensitive to initial states. This becomes more like guessing than prediction. It’s not ml, or AI, it’s delusional fairy gas.

  20. Pingback: reconstitute the world « Bethany Nowviskie

  21. Pingback: What machine learning means for software development – f1tym1

  22. Pingback: What machine learning means for software development | Tesco Inc.

  23. Pingback: What machine learning means for software development - O'Reilly Media

  24. Pingback: What machine learning means for software development – A.I WARE

  25. Pingback: 6 New Trends in Software Development

  26. Pingback: 对于软件研发而言机器学习意味着什么

  27. Pingback: Industry experts weigh in on the adoption of AI and ML in software development | Fingent Blog

  28. Pingback: 4 Ways AI (Artificial Intelligence) Improves Software Development

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: