Workshop Description

We increasingly rely on personal and professional advice generated and curated by algorithms, rather than friends and family.

In the first session of this workshop, we start by discussing the historical context of search and recommendation systems. Next, we delve into the mechanisms behind these systems, which heavily rely on state-of-the-art machine learning techniques. The process involves converting implicit and explicit user preferences into structured data that can be computationally analyzed to provide valuable recommendations. Additionally, we examine how search can be seen as a specialized form of recommendation.

In the second session of the workshop, we focus on the efficient implementation of these computations using contemporary distributed computing methods.

About the Instructor

Hao Sheng is a Staff Machine Learning Engineer at Apple SPG (Special Projects Group). He received his PhD in Computer Engineering from ICME, Stanford University as part of Stanford Machine Learning Group and Stanford Computational Policy Lab. His research interests are machine learning algorithms for environmental and social sustainability. He worked on large-scale recommendation systems at TikTok before joining Apple. Prior to Stanford, he earned his Bachelors in PPE (Philosophy, Political Science, and Economics) and a B.S. in Applied Mathematics from Yuanpei College, Peking University.

About the Teaching Assistant

Rui Yan is a Ph.D. candidate at ICME, Stanford. Her research focuses on developing machine learning algorithms to extract meaningful representations from large-scale, high-dimensional and multi-modal data. She has interned at Meta, Uber AI, and Microsoft Research.

Workshop Materials

Pre-workshop Checklist

The Zoom link to the course can be found here.

This advanced workshop is designed for experienced Python programmers, preferably with recent experience applying neural networks.

Familiarity with deep learning at the level of SWS 11: Introduction to Deep Learning, is expected.

Syllabus

Session 1 (Wednesday, August 9 | 8:00 AM - 11:00 AM PDT)

Day I: Recommendation Systems 101

• Introduction
• Lecturing: The History of Recommendation System
• Break-out: Recommendation System in Daily Life
• Lecturing: Recommendation as an ML Problem
  • User-item rating matrix
  • Collaborative filtering
  • Content filtering
• Lecturing: Evaluate Recommendation Systems
• Lab: Recommendation System Notebook I

Zoom recording Day 1: Link

Session 2 (Thursday, August 10 | 8:00 AM - 11:00 AM PDT)

Day II: Advanced Topics

• Recap
• Lecturing: Search Engine System
• Break-out: Measure the Success of the Recommendation System
• Lecturing: Advanced Topics (Part I)
  • Augment recommendation system with deep learning
  • Speed up and scale up
• Lab: Recommendation system notebook II
• Lecturing: Advanced Topics (Part II)
  • Recommendation with LLM (Large Language Model)
  • Social impact

Zoom recording Day 2: Link

Requirements

To join the workshop, you’ll need a device with a recent web browser and two-way audio and video access to Zoom. This could be a laptop or desktop computer running any operating system, such as Windows, Mac, or Linux. Participative activities benefit from a larger screen, so joining via a smartphone or tablet may not provide the best learning experience.

You do not need to install Python or any other software before the workshop. We will provide more detailed instructions prior to the start to ensure that you are set up and ready to learn.

Our goal is to create an inclusive and supportive learning environment, and we want all students to succeed. However, to set you up for success, we also want to clearly communicate the necessary level of prior knowledge and programming skill. If you are unsure whether you have the required background, please feel free to reach out for guidance.

Outcomes

Workshop attendees will be positioned to leverage their newfound expertise in search and recommendation systems to contribute to industry practices and make informed decisions on the implications of these increasingly prevalent technologies.

Specifically, attendees will:

• Develop a comprehensive understanding of the evolution of recommendation systems, along with their relationship to search systems. They will grasp the underlying deep learning mechanisms and techniques employed in these systems.
• Acquire practical engineering skills essential for efficiently implementing recommendation systems at scale.
• Learn skills for building recommendation systems for diverse applications, empowering them to contribute to the advancement of the field.
• Gain insights into the state-of-the-art practices of machine learning within prominent Silicon Valley companies that rely on search and recommendation systems for adding value.

References

A list of references at varying levels of sophistication are provided later in this description. Students interested in maximizing their learnings from this workshop will do well to familiarize themselves with as many of the references as possible.

• Schrage, Michael. Recommendation engines. MIT Press, 2020.
• Liu, Zhuoran, et al. “Monolith: Real Time Recommendation System With Collisionless Embedding Table.” arXiv preprint arXiv:2209.07663 (2022).
• Heng-Tze Cheng, et al. 2016. Wide & Deep Learning for Recommender Systems. Proceedings of the 1st Workshop on Deep Learning for Recommender Systems (2016).
• Paul Covington, Jay K. Adams, and Emre Sargin. 2016. Deep Neural Networks for YouTube Recommendations. Proceedings of the 10th ACM Conference on Recommender Systems (2016).
• Alexandra Egg. 2021. Online Learning for Recommendations at Grubhub. Fifteenth ACM Conference on Recommender Systems (2021).
• Gupta, Udit, et al. “The architectural implications of facebook’s dnn-based personalized recommendation.” 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 2020.
• Meituan. 2021. Distributed Training Optimization for TensorFlow in Recommender Systems (in Chinese). https://tech.meituan.com/2021/12/09/meituan-tensorflow-in-recommender-systems.html
• Wang, HaiYing, Aonan Zhang, and Chong Wang. “Nonuniform Negative Sampling and Log Odds Correction with Rare Events Data.” Advances in Neural Information Processing Systems 34 (2021): 19847-19859.