Hi !

Quick house-keeping note: the podcast will be taking a two week break (next episode will be released April 20). I've started training Jiu-Jitsu again, which is what producing the podcast originally had replaced, and I'm actively figuring out how to balance everything.

Onto the newsletter!

A Practical Approach to Learning ML with Radek Osmulski

Radek Osmulski is a fully self-taught machine learning engineer. After getting tired of his corporate job, he taught himself programming and started a new career as a Ruby on Rails developer. He then set out to learn machine learning. Since then, he's been a Fast AI International Fellow, become a Kaggle Master, and is now an AI Data Engineer on the Earth Species Project.

In this episode, Radek details his journey switching careers into software engineering and then into machine learning. He talks about mistakes he made, how he would do it now, and gives a preview of his forthcoming book.

Listen to the episode!

150 Successful ML Models at Booking.com

"Our main conclusion is that an iterative, hypothesis driven process, integrated with other disciplines was fundamental to build 150 successful products enabled by Machine Learning."

This KDD'19 paper from Booking.com is chock-full of practical advice on developing ML-driven features and products. It covers lessons learned in how they use ML to create business impact, problem framing and selection, model monitoring, and evaluation experiment design.

This is the paper that we'll be discussing tomorrow in my reading group--if you haven't signed up for that already, you can do so here.

Can data cleaning be taught?

A recurring theme in the newsletter and the podcast has been the importance of using clean data as input to your models. But the skill of data cleaning, much like feature engineering, remains more of an art than a science, a tacit knowledge that is obtained through experience rather than taught.

In Randy Au's latest newsletter, he explores this question in detail. It's highly relevant for anyone who doesn't consider themself yet proficient in this dark art (🙋‍♂️).

GPT-Neo Models Released on 🤗

The ML open-source community continues to over-deliver as the two GPT-Neo models from ELeutherAI were released and are now available in the HuggingFace library and model repo.

If you haven't been following their work so far, GPT-Neo is an open-source implementation of GPT-3. So far they have trained models with 1.3B and 2.7B parameters (compared to 175B of GPT-3), and are actively working on scaling up even further.

Also see their previous work on The Pile, the massive text dataset these models were trained.

MIT Tech Review Interview with Andrew Ng

"I’ve gone to many of my teams and said, “Hey, everyone, the neural network is good enough. Let’s not mess with the code anymore. The only thing you’re going to do now is build processes to improve the quality of the data.” And it turns out that often results in faster improvements to performance of the algorithm."

Last week, I highlighted Andrew Ng's talk on MLOps, and this week I came across an interview he did at EmTech Digital. He talks about why he doesn't think companies should be "AI-First", the merits of good data over big data, and his process for evaluating opportunities to solve problems with ML.

Thanks for reading and have a great rest of your week!

Charlie