Summary
In season 2 Ep. 7 of the Dr. Pawd podcast, the host interviews Ross Whiteman, an independent researcher and contributor to the AI/ML open source movement. Ross left his job at a Canadian startup to become an angel investor and start tinkering with AI. He learned about AI through Kaggle, a data science competition platform where participants compete to solve problems related to machine learning and data analysis. Ross eventually started working on his own project, the PyTorch Image Model Library, which collects models for different vision-based challenges on Kaggle.
The podcast discusses the importance of open source contributions and how they can have a significant impact in the field of AI and deep learning. The speakers talk about the merits of using smaller and open accessible data sets like Imagenet 21k and how it's better for transfer learning to use more data than to try and train on Imagenet 1k by cranking up augmentations and regularization. The episode highlights the importance of combining different ideas from individual papers and testing them in combination is a trend in AI research that can lead to greater progress as a community.
Ross describes his challenges with running experiments that require larger data sets and more compute, and how he overcomes these challenges by accessing the TF Research Cloud and TPU resources from Google, adding support for PyTorch XLA, and receiving support from organizations such as Hugging Face, PyTorch Lightning, and Oldify. He also discusses his preference for being an independent researcher, stating that he likes to be able to choose what he works on each day and move between different projects and ideas.
The guest spends most of his time developing and interacting with organizations using the models. He has a 19-month-old toddler who takes up most of his time outside of work. Besides work and family, the guest also enjoys hiking, skiing, and investing in angel companies. The speaker recommends Kaggle as a way to get started with AI and to get involved in the community.
Overall, the episode offers a wide range of insights into the AI/ML open source movement, including the importance of open source organizations and contributions, the value of different data sets and competitions, and the challenges and joys of working as an independent researcher. It's a must-listen for anyone interested in the field and looking for inspiration and guidance on how to contribute to the community.
