Yohan Lee is the Chief Strategy Officer at Riiid Labs, a leading solutions provider of fully verified and data-driven AI technology. They partner with global leaders in education, skills training, and technology to create better learning experiences. They use AI to help learners realize their goals in the fastest, most efficient way.
What initially attracted you to AI?
Riiid’s focus. Riiid had made the right moves that a serious company with great potential would make. Each step was reasonable and inspiring. For its AI, the company started reasonably with collaborative filtering methods, then regularly improved their algorithms up to state-of-the art Transformer models with their own unique evolutions. The company kept publishing thoughtful papers that showed focus, exploration, and imagination. The company’s app was converting users and monetizing successfully which indicated strong market adoption. The business was expanding from a B2C to a B2B dimension with additional test prep tools and a Realtor app. Then it published the largest education dataset to initiate greater innovation to create an AIEd community (EdNet). These are all moves of a focused company which was making the right moves that only a large company typically makes. Lastly, their publicly visible funding rounds were a strong reflection of the investment potential of how quickly this company could become a unicorn.
In your previous role you were a Machine Intelligence and Health Data Science Leader at Google Brain, what were some of the projects that you worked on?
I led Product Deployment: which is how to design, construct, measure, system test beds for real world evidence studies. A combination of advanced test bed construction, research strategy, and advanced systems design and implementation. I am not an engineer, but I’ve been a certified pro cloud architect on multiple clouds for a number of years now. So it was a natural fit, especially with the many information security and privacy requirements for PHI, and vulnerable populations (namely children).
Many times, there’s a technical readiness mismatch between partners and tech giants to reach scale. That’s where I drive scale. Making self-assembling tech infrastructure designed for ML engines to cost 1/10 the price and reach massive parallel compute at latency and cost.
Making prediction algorithms available to our partners as AIaaS, has been exciting.
Since August 2020 you’ve been Vice President of Strategy at Riiid Labs, could you explain what Riiid Labs is precisely?
Riiid Labs is a global leader in AI solutions for education. Riiid Labs is the global arm of its parent company Riiid and has been established in Silicon Valley to build upon Riiid’s success in Asia and to expand its business across the U.S., South America, the Middle East, and beyond. We are composed of AI researchers, data scientists, engineers, and business development and work with partners in various verticals to rethink the traditional ways of learning via extending Riiid’s AI competency. Whether it is a personalized test prep mobile app for the college entrance exam in South America or an AI tutor module for training insurance agents at a major conglomerate in Korea, we offer B2C/B2B/B2G customers with our proprietary technology in AI that analyzes user’s learning behavior data, predicts next answer choices, and recommends personalized learning path that ultimately maximize one’s learning potential. We have launched several products already including Santa, a mobile test prep application for the popular English proficiency exam, Test of English for International Communication (TOEIC) which has been used by more than two million students in Korean and Japan. We have also released a GMAT prep app in Korea based on partnership with Kaplan and an ACT prep app in Egypt, Turkey, UAE, Jordan, and Saudi Arabia in partnership with ConnecME. Riiid is now in talks with a wide array of customers from private and public sectors to implement our AI solution for educational assessments, learning and training.
How are AI tools a step forward in democratizing education?
According to the UN, even before the coronavirus crisis, projections showed that more than 200 million children would be out of school, and only 60 per cent of young people would be completing upper secondary education in 2030. Half the developing world’s children left education without any applicable qualifications for the workplace. ‘One size fits all’ approaches to education and high stakes standardized test based assessments have failed to satisfy the expected role of education to grow talented citizens of society.
In the developed economies such as Korea and the US, wealthy communities have much larger school budgets than poorer communities. Subsequently, wealthier parents make additional investments in private tutoring, private college advisement, test prep, private athletic coaching, and out of school learning experiences in the arts and culture, that widen the gap between their children and everyone else. This means that students in less wealthy communities often have less experienced teachers, less access to technology, reduced internet access at school and at home, and may not receive any advice about applying to college. In the US, a child born into a wealthy family is 10X more likely to complete a college degree than a child born into a poor family. And even when you hold academic ability constant, the wealthier child is much more likely to go to college and to complete a degree. This also has implications for economic growth. When talent and potential are widely distributed in a society, but opportunity is not, the labor market and economy cannot efficiently match talent with employment, and that slows down innovation, and compromises national productivity and economic growth.
Based on significant data, AI can assess and understand each student’s knowledge level and unique learning behaviors and provide personalized content timed to help students reach every learning goal. AI is eternally patient and can give everyone equal yet individualized attention at a fraction of the cost of personal tutors. It is possible that any student with an Internet connection and smartphone will be able to participate in an engaging, personalized learning experience regardless of where they live. AI can also help teachers personalize the learning experiences of their students, reducing the time teachers must spend on repetitive tasks, and repurpose that time to provide individual attention and personalized learning resources available to every student 24/7, regardless of whether school is open or not.
Could you discuss the vision behind the first-ever global Artificial Intelligence Education (AIEd) Challenge?
The world needs a new paradigm in education to overcome the current crisis in education. AI-powered learning solutions that students interact with online can give everyone equal yet individualized attention at a fraction of the cost of personal tutors, whether used for independent learning or incorporated into teacher-led learning experiences. We truly believe our vision in AI education and are certain we can transform education and improve the lives of students. But we know we can’t achieve this vision alone. We need the larger AI community and education industries to buy into our ideas and participate as well. Last year, Riiid publicly released EdNet, a large-scale hierarchical dataset of diverse student activities collected from Riiid’s AI tutoring system. It contains data from more than 131 million interactions with more than 780K real-world students. It is the largest among the AI-education datasets released to the public so far. We want the best minds in the field to use this data to find innovative solutions that would help tackle the global challenges in education. Hence, Riiid launched its inaugural AIEd Challenge, a global challenge to create and evaluate algorithms for knowledge tracing using EdNet. We believe that by joining forces we can make a bigger impact and further accelerate the trend toward AI-enabled education. Leading by example is necessary for leadership and inspiration to others.
What were some of the results from AIED?
The Challenge ran from Oct 6, 2020 to Jan 8, 2021 via Google’s Kaggle platform, an online community of data scientists and machine learning practitioners.
- For the total prize of $100,000, 3,395 teams from 90 countries participated in the Challenge, making it the most of any 2020 Kaggle algorithm competition hosted by a business entity.
- 52 of Kaggle’s 270 Grandmasters participated, the highest proportion for Kaggle competitors based on past performance. In comparison, 2020 competitions averaged only 25 Grandmasters in participation.
- Through this challenge, 64,678 different and creative Knowledge Tracing models were submitted.
- Teams from Korea, Japan and Spain won the top three places, receiving $50,000, $30,000, and $10,000, respectively. The top three teams presented their models at the AAAI-2021 Workshop on AI Education: Imagining Post-COVID Education with AI, hosted and organized by researchers at Riiid.
- All the winning solutions used Transformers, an attention-based model first introduced with uses in machine translation [AAYN] (Vaswani et al) and adopted by Riiid researchers. This shows that the value of Transformers is quite clear, the innovative ways the Kagglers used them were both academically and practically interesting. These were creative applications of Transformers that were unexpected by our researchers. We were so encouraged that providing this kind of platform could promote such diverse experimental approaches in AI Education. We believe this will lead to the advancement of applicable technology as an extension of this solid foundation for research.
How can AI best accelerate personalized learning?
In summary, AI technology, based on deep learning algorithms, analyzes user data and content and predicts scores and behavior. Based on these insights, AI recommends personalized study plans in real-time.
And Riiid’s core AI technologies do just that: 1) Knowledge Tracing, 2) Score Prediction, 3) Personalized Recommendation.
- Knowledge Tracing: Knowledge tracing is one of the fundamental tasks in the AI-Education field. Discerning what students know and do not know at a point in time provides the baseline for constructing the optimal learning path. And the model is able to predict correctness of a learner for any and all unsolved questions. Our deep learning based Knowledge Tracing model inspired by the Google Transformer predicts whether a student will respond correctly or not to a question with the highest accuracy.
- Score Prediction Model: The Score Prediction model predicts a student’s level of achievement during the learning process. Our model predicts a student’s score with ± 5% mean prediction error, which could be broadly expressed as 95% accuracy. Assessment gives feedback on a student’s knowledge state, which enables real-time learning adjustments. And it allows students to recognize their progress and achievement which promote self-assessment and practice.
- Recommender System: Based on Knowledge Tracing and Score Prediction models, we provide learners with items required for the most improvement. Designing a recommendation system isn’t always straightforward, and it requires a lot of consideration and research into understanding what is best for the learner. Riiid developed Recommendation for Effective Standardized Exam Preparation (RCES), an AI model that recommends questions which do not just maximize Expected Scores but makes sure learning takes place. It avoids the goal of solely trying to increase test scores in the absence of true learning by reflecting the impact of acquiring new knowledge from the solutions of incorrectly responded questions.
Could you discuss the Riiid Santa apps and what users should expect?
Riiid offers Santa, a mobile test prep application for the popular English proficiency exam, Test of English for International Communication (TOEIC). The app features Riiid’s proprietary AI technology which analyzes user data and content, predicts scores and behavior, and recommends personalized study plans in real-time to help users optimize their learning potential. The app has been used by more than two million students (2.5M) in Korea and Japan and reached No. 1 in sales among education apps in Japan and Korea. Based on user data over a year, the average score increased by 165 points out of a possible 990 points after just 20 hours of study.
Is there anything else that you would like to share about Riiid or Riiid Labs?
With just six years under our belt, we have only scratched the surface of the limitless prospects in AI Education. We have seen how we can change the way people take so-called traditional learning approaches for standardized tests. But we also saw a possibility of a more meaningful measure of a student’s capabilities. And we truly believe we are best equipped to lead this revolution.
Our belief is that only through understanding the student’s daily learning behavior and path, can we assess a student’s true potential and optimize their learning experience accordingly. This wasn’t possible before. But with AI, we can analyze the current level of the learners and instantly personalize instruction that fits an individual’s needs. We can constantly motivate them and keep track of their progress. We can allow every learner to grow in his own way, in his own pace, which the current system does not stand for. With this approach, we can truly unlock one’s learning potential.
Our vision is only being realized with test-prep services but we are at the beginning of something great. Summative assessment, which evaluates learners through one-off tests, is not the best tool for assessment and effective learning. And further, it is now physically invalid due to the Covid19, and we saw the huge market needs in effective and practical formative assessment tools. That’s why Riiid is now working on a deep learning based, domain-independent ‘capability measurement model’ called “Riiid-Score.” We are confident that we will transform the core of education by supporting formative learning, which refers to the process of constantly optimizing the teaching-learning experience. We believe this to be a basic step towards education that can unleash the potential of every individual.
Thank you for the great interview, readers who wish to learn more should visit Riiid Labs.
- Do Conversational Agents Like Alexa Affect How Children Communicate?
- Hobbling Computer Vision Datasets Against Unauthorized Use
- Faisal Ahmed. Co-Founder & CTO at Knockri – Interview Series
- The Shortcomings of Amazon Mechanical Turk May Threaten Natural Language Generation Systems
- AI Chipmaker Deep Vision Raises $35 Million in Series B Funding