Sivasubramanian lead off with a talk about machine learning being “one of the most transformative” technologies in a generation. He cited a stat that more than 100 papers in machine learning are published each day. “Machine learning is going mainstream,” said Sivasubramanian. More than 100,000 customers use AWS for machine learning, said Sivasubramanian, citing examples such as pharma giant Roche and The New York Times. 
Sivasubramanian offered examples of “working backward from the customer” in the development of ML. The first example was how a system can “learn with less data.” Accessing and annotating data is “too tedious” as ML becomes mainstream, said Sivasubramanian. He cited the example of the NFL wanting to manage its library of video assets from football games. Another customer, an 80-year-old pizza company, wanted to ensure every pizza has the same amount of cheese to maintain quality. The company used AWS to get an imaging system for pizza inspection. The solution was what’s called “few-shot learning,” where machine learning is supplied with only a limited number of examples. 
Sivasubramanian cited the desire to replicate a real factory setting. So, the team that developed Lookout for Vision built a replica of a factory to try the few-shot approach in the real world. 
Sivasubramanian’s next example was understanding “irregular text” with machine learning, where, for example, text is blurred. Accuracy goes way down, he noted. That’s important for real-world instances such as transcribing doctors’ handwritten notes.
The traditional language model approach of guessing with the first few letters of a text runs into problems when there is little context. So, the AWS team invented something called “SCATTER,” for “Selective Context Attentional Scene Text Recognizer.”  SCATTER sends an image through additional processing that has a decoder that can choose to employ either contextual or merely visual information. The SCATTER technology lead to a 3.7% improvement in text recognition on benchmark character recognition tasks, a big improvement, Sivasubramanian said. SCATTER is now used in AWS’s automatic text extraction service.  
Sivasubramanian then brought up Maarek, to talk about “giving Alexa a sense of humor.” Maarek referenced Alan Turing’s 1950 paper on “Computing Machinery and Intelligence,” in which the mathematician argued against presumptions about computers. 
“Think of debuggers,” said Maarek, which is an example of how a computer “thinks about its own thought,” something people thought computers wouldn’t do but that Turing said they would. “Already, Turing was looking at having a sense of humor being a really hard challenge” in computer science, said Maarek.   Rather than trying to make Alexa be funny, said Maarek, the challenges was, “We want to look backward, ask whether customers are funny, and how should the machine respond to it,” explained Maarek.  Maarek said the team built a deep learning model, employing notions about humor such as subjectivity, and using embeddings. “We took into account domain bias, to make sure we didn’t over-fit our model.” As a result, the team was able to present a paper with high degrees of humor accuracy at last year’s SIGER conference. 
Then, the team moved on to how to detect with speech in Alexa. Would customers appreciate, she asked, Alexa understanding the humor? Or did customers want to feel superior? Maarek cited humorous user utterances toward Alexa, such as “Alexa, can you burp?”  “You will see a ton of toilet humor,” she said, “It’s part of a very important area of humor, relief humor.”  “Alexa, what is your blood type?” was among the things that get asked. Some examples, she noted, are not so much funny as playful. Such utterances are examples of both personification and superiority on the part of humans. “We defined playfulness,” she said. Playfulness means, “the customer doesn’t expect Alexa to take this request literally,” and Alexa should not add anything to the shopping list of the user.
Maarek said she and the team had to go back to researching the teachings of Aristotle, Kant, Schopenhauer, and other great thinkers regarding humor, to understand all the forms of humor. Surveying all the forms of humor helped the team understand the matter of what users will enjoy from Alexa. The question became, Will users enjoy it if Alexa understands their humor? 
The team started with “personification,” where people relate to Alexa as a personality, as a point of conjecture to explore the problem. They recruited a hundred college students in a blind question-asking exercise, talking to an entity they didn’t know was Alexa (The entity was named “Shirley,” a play on the movie Airplane.)  The students’ questions were examined by a custom version of Google’s BERT transformer neural net. It employed sentiment analysis and such. “We got a pretty good model,” she said, “to detect these funny personification utterances on the fly.”  The team went to a speed-dating site to scope out questions people ask when trying to be funny or playful. That lead to a survey of personification questions that people ask “Do you think as good as a woman?” is one kind of question that gets asked.  The result was that the team determined that human questioners enjoy it when Alexa responds to their playfulness. “They really want to have fun not at Alexa but with Alexa,” was the conclusion of the research.
Sivasubramanian moved on to talk about “horizontal” use cases, where customers don’t have much in the way of ML skills. That includes “embedding” what’s called “autoML,” where customers don’t need to know about model design or tuning. The tech is then used for things such as customer service, document recognition, etc. Sivasubramanian called out domain-specific models for healthcare, such as medical note transcription.
Then Sivasubramanian moved on to deploying machine learning at scale, and he invited up Saha. Saha made the point that customers have increased their model deployment from “just a few” models in the early days to “thousands” per customer. SageMaker, he noted, now supports hundreds of billions of predictions per month. “From a dozen models to millions of models and hundreds of billions of predictions in just a couple years,” was how Saha summed up the progress. 
Saha cited Lyft as a customer. They used SageMaker to reduce model training time for “Level 5” ADAS (self-driving.)
iFood, a leading food delivery company in LatAm, used SageMaker to reduce the travel distance of delivery staff. 
“We are building SageMaker along three vectors,” said Saha. “Infrastructure, tools, and ML industrialization.” Saha talked about things built on top of SageMaker, including AWS Inferentia, which is used by customers such as Snap, Autodesk, and Condé Nast to lower their cost and increase their performance of inference. 
Saha talked about the Habana Gaudi-based chips that will be coming to EC2 instances this year (Habana is a unit of chip giant Intel). He also called out AWS’s own home-grown “Trainium” chip for ML training, also coming later this year. 
Also: AWS’ custom chip family expands, launches Trainium for machine learning models Saha moved to talking about the problem of deploying multiple endpoints, one for each ML model. The solution was SageMaker’s “multi-model endpoints,” which allows one to “host hundreds of thousands of models on a single endpoint.” That leads to optimization of prediction accuracy and throughput, he said. 
A third vector, he said, was industrialization of machine learning. “We asked ourselves, How did software go from a niche to an industry?” It involves tools from software such as IDE and CI/CD. SageMaker is the first IDE for ML, he noted. Another analogous tool is the CI/CD tool, something that is rarely available in ML, he noted. 
ZDNet’s Larry Dignan chatted with Ng and Saha about the course. 
Sivasubramanian came back on to introduce Intuit’s chief data officer, Srivastava. Srivastava said Intuit’s “mission” is “powering prosperity around the world.” He cited stats of tax returns filed via TurboTax (48 million) and “Mint users empowered to make smart money decisions” (over 25 million.)
Also: AWS, DeepLearning.ai aim to bridge scaling gap with machine learning models via Coursera specialization Srivastava used the example of TurboTax Live, which poses questions about a person’s work and such to figure out one’s taxes. “Through the magic of the virtual expert platform, you can get connected to the right expert,” a human consultant, he said. That is made possible by machine learning that does routing, he noted. 
Sivasubramanian, closing out the keynote, noted in broad sweeps how machine learning is “transforming entire business processes.” That’s happened by “barriers to entry being lowered.” 
And that’s a wrap!