How to break into machine learning

broken_glass

Photo by Erich Ferdinand

An engineer recently asked me how she could turn an interest in machine learning into a full-time job. This can be a daunting prospect, because the whole field has until recently been very separate from traditional engineering, with only a few specialists at large companies using it in production, often far from traditional product teams. I took a very random path to focusing on deep learning full time, but so did most of the people I work with. It’s not clear that there is one good route, but I wanted to share the advice I had to offer in case it’s helpful to others.

Become a Designated Machine Learner

Every manager should point at one member of their team and say “You are now our machine learning expert”. If your manager doesn’t do that for you, announce it yourself to anyone who will listen. This may sound like madness, but machine learning is rapidly invading almost every product area, so whether you’re in games or enterprise software, your group needs to at least stay up to date with what’s happening with the technology. If you aren’t, then your competitors are!

You may have to fight your own imposter syndrome, but becoming the go-to person for everyone’s questions about machine learning is a fantastic way to teach yourself the essentials. You’ll have to say “Good question, let me go figure that out” a lot at first, but every expert I know does the same! Even if you don’t end up building anything in production, at least you’ll be able to point at relevant research and experiments if you decide to change to a new position.

Enter Competitions

I have been a massive fan of Kaggle since it got off the ground. If your job’s not offering you the opportunities in machine learning you want, then joining that community is a great way to teach yourself a lot of practical skills. If you look through the forums, a lot of the contestants will describe exactly how they solved old competitions, so I would recommend following a few of their recipes to get started. Once you’re able to do that, pick a new contest that’s similar to one of those, and start playing around with all of the different options to see how you can improve the results. Most of machine learning is the software equivalent of banging on the side of the TV set until it works, so don’t be discouraged if you have trouble seeing an underlying theory behind all your tweaking!

Find a Community

As I mentioned above, the most frustrating thing about machine learning is how arbitrary it all is. I’m lucky enough to be at a large company surrounded by people I can talk to about things like why my model isn’t learning, but most engineers don’t have that luxury. That’s another advantage of Kaggle, from what I’ve seen their forums offer a lot of support and encouragement. I would also look out for real-world meetups where you can swap stories and commiserate. If you can’t find something related to your field, try starting a mailing list or group yourself, or propose a session at a conference.

There is a long tradition of mentorship in machine learning, especially around deep learning, but I think we should be doing a lot better job of capturing all that oral tradition. As someone who was recently an outside myself, I want to see the field democratized. I think the reliance on word-of-mouth is more about poor written communication than anything inherent in the subject.

Write Documentation

On that topic, my TensorFlow for Poets post came out of work I was doing to help myself understand how to reliably retrain the top layer of a deep network. I didn’t know how before I started, but by carefully documenting the process and making sure I could reproduce it consistently, I learned a lot about how it all works. I also got a lot of helpful feedback as I shared drafts of the guide with colleagues.

One interesting thing about human nature is that people are a lot more willing to correct somebody else’s mistaken ideas than they are to propose their own. As long as you’re happy to keep eating humble pie, that means writing up your own tentative understanding and getting it reviewed is a lot more effective way of getting others to share their knowledge than asking flat out! That’s another reason I try to do documentation, purely for the corrections.

Don’ts

Unless you’re doing a degree at a recognized university, I personally don’t recommend going for a credential in machine learning. I do love courses like the Udacity Deep Learning program, but for the content not as a resumé builder. Having practical experience, even just on competitions like Kaggle, will be a lot more helpful in interviews.

As an engineer, I also find many machine learning research papers hard to get much benefit from. They tend to assume a lot of prior knowledge from the academic world, and prefer presenting their ideas in math rather than code. They can be useful once you’re experienced, but don’t worry if you’re left baffled by them at first.

Anyway, I hope some of these ideas are useful. Definitely read them with a skeptical eye, nobody really knows anything in this field, and I’ll be interested to hear what other suggestions people have!

8 responses

  1. Whoah, thanks for this post!

    I’m just about finished Andrew Ng’s machine learning course then moving onto Udacity’s next. Maybe after that I’ll feel up to Kaggle. Not even sure if this is something that I want to do but hell it’s held my focus and interest this long.

    I personally feel the lack in local community involved with machine learning, mainly due to living in a small city. So far the only local people I’ve found who understand machine learning lingo are Math Professors! So if anyone out there want’s to talk ML with a newbie please add jkfrancis14 to Skype.

    Cheers!

  2. Huzzah! These articles are great timing for me, just getting a huge chunk of time where I can finally focus on the neat things going on in ai (for the last few decades). Yes, the math is intimidating, but hopefully I’ll get my errors down and really learn it, have a hefty set of hidden layers and quantized weights for the subject.

    Also I’m concentrating on Python with 2.7 and TensorFlow (And a bunch of other misc supportive imports and libraries). I think will get me on the way.

  3. Pingback: Ruta para aprender Machine Learning | blog.greenvalley.es

  4. Very good article! I feel that the main difficulty lies in the fact that the field of ML has been evolving so fast that if one were not in one of those big name companies, it is really hard to keep track all those progresses, as an outsider in the beginning.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: