Xiaofeng Ren at AI Frontiers: The Quest for Video Understanding

•

2 likes•1,680 views

In this talk I will briefly discuss the ubiquitous needs of video and video understanding across Alibaba and the challenges that are being addressed and solved at iDST, Alibaba's AI R&D division. Examples include mobile shopping on Taobao, video search and recommendation on Youku and Tudou, and real-time systems for Cainiao Logistics and City Brain.

Data & Analytics

The Quest for Video Understanding
Xiaofeng Ren
Institute of Data Science and Technologies (iDST), Alibaba Group

Alibaba Ecosystem
3
CLOUD COMPUTING OPERATING SYSTEM
DATATECHNOLOGY TEAM
Online and Mobile Commerce Platform
$
BUYERSSELLERS
Marketing
Financing &
Payment
Solution
Professional
Service Providers
Digital
Entertainment
Logistics
Mobile Browser
Location-BasedSocial Media
Weibo

Alibaba Mission
4
CLOUD COMPUTING OPERATING SYSTEM
DATATECHNOLOGY TEAM
Online and Mobile Commerce Platform
$
BUYERSSELLERS
Marketing
Financing &
Payment
Solution
Professional
Service Providers
Digital
Entertainment
Logistics
Mobile Browser
Location-BasedSocial Media
Weibo
make it easy to do business anywhere

Taobao Search: Text  Text
9
Title：茜舞2017春装新品韩版修身公
主裙红色连衣裙无袖打底背心裙春秋
Q：无袖蓬蓬裙连衣裙夏 a型
Title：美特斯邦威卫衣女士
春装字母拼色针织套头衫
Q：百搭灰色套头无帽卫衣女

Taobao Search: Text  Text
10
Title：茜舞2017春装新品韩版修身公
主裙红色连衣裙无袖打底背心裙春秋
Q：无袖蓬蓬裙连衣裙夏 a型
Title：美特斯邦威卫衣女士
春装字母拼色针织套头衫
Q：百搭灰色套头无帽卫衣女

Taobao Search: Text  Image
11
Title：茜舞2017春装新品韩版修身公
主裙红色连衣裙无袖打底背心裙春秋
Q：无袖蓬蓬裙连衣裙夏 a型
• Joint text and image embedding for description
• Leverage user click data to learn query-description similarity

Taobao Search: Image  Image
12
3 billion images over 100 million products, 12 million daily active users

Videos on Taobao
14
• Promotional video
• Video guide on how to use product
• Documentary on how a product is made
… …

Live Streaming on Taobao
15
• Many top-selling stores have live streaming
• Allow user interaction (chat, FAQ)
• Conversion rate much higher than normal

Videos on Taobao
16
• Short videos are abundant on Xianyu (second-hand trading platform)
• Demonstrations of products; sometimes people showing skills
• Much richer information than text descriptions

Videos on Taobao
17
• Authoring tools
• Video enhancements
• Content analysis and tagging
• Video-to-text matching
… …

Youku & Tudou: “Youtube-like”
19
Founded 2006
Founded 2005
IPO 2010
IPO 2011
Merged
2012
Joined Alibaba
2016

Youku: Video Hosting
20
30M paid members
~400M active users
700-min per month

Tudou: UPGC Sharing
25
Copyright protection
Authoring tools
Search & recommendation
… …

Cainiao Logistics
27
24 hours in China
72 hours globally
1B packages on 11/11Mission:

Robots @ Cainiao
Delivery Robots Delivery Drones
Picking Robots

Sample Video Applications
30
Digital
Entertainment
Logistics
CLOUD COMPUTING

Videos Everywhere
31
CLOUD COMPUTING OPERATING SYSTEM
DATATECHNOLOGY TEAM
Online and Mobile Commerce Platform
$
BUYERSSELLERS
Marketing
Financing &
Payment
Solution
Professional
Service Providers
Digital
Entertainment
Logistics
Mobile Browser
Location-BasedSocial Media
Weibo

DAMO Academy (including iDST):
$15B investment on R&D to solve “big problems”

What's hot

Imagimob presentation at the Stockholm IoT Investment Summit

Anders Hardebring

Winning the Cognitive Era with Watson IoT - Sanjay Brahmawar

SITA

Lesser Known Opportunities in Technology

Orange and Bronze Software Labs

Lesser Known Opportunities in Technology

Calen Legaspi

Ibm ai in cognitive era

Mike Chang

Decision points for when to implement automatic indexing or more intensive subject analysis Machine learning and artificial intelligence approaches to automatic indexing and other aspects of content enrichment have tremendous potential, but there are significant barriers to successful implementations. The economics of these systems are not now generally affordable, which will indefinitely delay widespread adoption. Significant costs are involved in just the training and maintaining systems that chronically under perform and are fail to scale. Cost and performance data will be characterized and presented. Machine learning and artificial intelligence projects are not for the faint of heart, nor for those with small budgets. Key cost elements are identified along with approaches to estimating costs based on actual and reported cases.

IC-SDV 2019: The Economics of Artificial Intelligence and Machine Learning fo...

Dr. Haxel Consult

Cloud, Big Data, IoT, ML - together to build a real world use case!

Krishna-Kumar

BTO2017 | TEN - Olaf Slater - SABRE

BTO Educational

A recurrent topic in patent landscaping, monitoring and analysis is the necessity to operate in a defined technology field. The definition of such a field is, however, not always straightforward due to the ever increasing data volume and complexity. In addition, a traditional, classification based approach is often insufficient to encompass the relevant data in a growingly interdisciplinary context. A number of predefined, precision rather than recall oriented technology fields is being developed by ip-search and rendered available within the PatentSight business intelligence platform. Further specificity can be achieved, if desired, via the application of a machine learning system. A case study is presented.

IC-SDV 2019: Anna Maria VIlla: Technology fields supported by machine learnin...

Dr. Haxel Consult

Real time ai on the edge

Anders Hardebring

The Human Dimension of Industry 4.0

Andy Fawkes

BTO2017 | TEN - Claudio Fadda - IBM

BTO Educational

IoT

Mphasis

Automatski - Video Surveillance

automatskicorporation

Internet of Things (IoT) assumes a critical part in the advancement of different fields. The IoT data trusted exchange in recent year extend of uses influence an awesome request and increasing scale. In such a platform, exchange the data sets that they require and specialist organization can search. However, the enough trust as the third-party mediators for data exchange in centralized infrastructure cannot provide. This paper proposes a blockchain for IoT data trusted exchange based on decentralized solution. In particular, the fundamental standards of blockchain in verify manner, individuals can communicate with each other without a confided in mediator intermediary. Blockchain enable us to have a distributed, digital ledger. IoT (Internet of Things) sensor devices (zigbee) utilizing blockchain technology to assert public availability of temperature records, tracking location shipment, humidity, preventing damage, data immutability. The sensor devices looking the temperature, location, damage of each parcel during the shipment to completely guarantee directions. In blockchain all data is got moved from one position to another, where a smart contract assesses against the product attributes. Ethereum blockchain and smart contracts atlast it gets through knowledge a design to be copied and presents its decentralized distributed digital ledger, auditable, transparent, features visually.

IoT–smart contracts in data trusted exchange supplied chain based on block ch...

IJECEIAES

Top Internet of Things Predictions for 2017

Brian K. Buntz

Ppt ATD 24th_aug_final

Kanika Pandey

Ubiquity e Autonomy : the autonomous car and the outcome economy

Roberto Siagri

Automatski - Open City

automatskicorporation

IoT

Mphasis

What's hot (20)

Imagimob presentation at the Stockholm IoT Investment Summit

Winning the Cognitive Era with Watson IoT - Sanjay Brahmawar

Lesser Known Opportunities in Technology

Ibm ai in cognitive era

IC-SDV 2019: The Economics of Artificial Intelligence and Machine Learning fo...

Cloud, Big Data, IoT, ML - together to build a real world use case!

BTO2017 | TEN - Olaf Slater - SABRE

IC-SDV 2019: Anna Maria VIlla: Technology fields supported by machine learnin...

Real time ai on the edge

The Human Dimension of Industry 4.0

BTO2017 | TEN - Claudio Fadda - IBM

IoT

Automatski - Video Surveillance

IoT–smart contracts in data trusted exchange supplied chain based on block ch...

Top Internet of Things Predictions for 2017

Ppt ATD 24th_aug_final

Ubiquity e Autonomy : the autonomous car and the outcome economy

Automatski - Open City

IoT

Similar to Xiaofeng Ren at AI Frontiers: The Quest for Video Understanding

Web Real-Time Communications (WebRTC) has been rapidly growing for the past 3 years gaining adoption from many corporations and industries around the world. Surpassing the one billion mark for compatible devices, WebRTC has sparked some heated debate around whether or not it’s a truly disruptive technology. In this presentation we will discuss the possibilities and the current applications built and running full WebRTC services, despite the lack of support in Internet Explorer and Safari and the codecs wars.

WebRTC: The Big Debate - I Say Shut and Build Something!

Blacc Spot Media, Inc.

T3CON14EU - The key to success - Generating relevance for TYPO3 Neos

die.agilen GmbH

Tracxn Research - Online Video Landscape, August 2016

Tracxn

Global Innovation Strategy of Tencent Holdings

jammygan

Whoscall 的 Realtime Monitoring 經驗分享

William Yeh

China Company Introductions 2008-03-20

dorrit

Understanding TikTok & ByteDance

Matthew Brennan

Kaltura - NOAH18 Tel Aviv

NOAH Advisors

Key issues in the development of publishing

ptslides

Making Video Communications and Training Searchable and Accessible

3Play Media

How APIs Are Reshaping The Future Financial Services

Stephane Dubois

APIs Are Powering Fintech Innovation. What Is Next?

Stephane Dubois

The Cloudification of Capital Markets

Stephane Dubois

Entrepreneurs' Organization NZ - Craig Rispin Keynote - 2 Aug 2016

Craig Rispin

Telco Global Connect 3

Sadiq Malik

IT Event Report of TC Disrupt/GMIC

01Booster

The Dragon Loves Online Video - Why Online Video Is Big Now..But Will Become ...

Richard Matsumoto

Day 1 C2C - Huawei - Video Strategy in Ultra-Broadband Era

Myles Freedman

(English Version) Gridow Business Overview-160520

Rich Li-Chi Huang

Infiltrating Telecoms Using Ruby

Mojo Lingo

Similar to Xiaofeng Ren at AI Frontiers: The Quest for Video Understanding (20)

WebRTC: The Big Debate - I Say Shut and Build Something!

T3CON14EU - The key to success - Generating relevance for TYPO3 Neos

Tracxn Research - Online Video Landscape, August 2016

Global Innovation Strategy of Tencent Holdings

Whoscall 的 Realtime Monitoring 經驗分享

China Company Introductions 2008-03-20

Understanding TikTok & ByteDance

Kaltura - NOAH18 Tel Aviv

Key issues in the development of publishing

Making Video Communications and Training Searchable and Accessible

How APIs Are Reshaping The Future Financial Services

APIs Are Powering Fintech Innovation. What Is Next?

The Cloudification of Capital Markets

Entrepreneurs' Organization NZ - Craig Rispin Keynote - 2 Aug 2016

Telco Global Connect 3

IT Event Report of TC Disrupt/GMIC

The Dragon Loves Online Video - Why Online Video Is Big Now..But Will Become ...

Day 1 C2C - Huawei - Video Strategy in Ultra-Broadband Era

(English Version) Gridow Business Overview-160520

Infiltrating Telecoms Using Ruby

More from AI Frontiers

As video content is becoming mainstream, video summarization is becoming a hot research topic in academia and industry. Video thumbnail generation and summarization has been worked on for years, but deep learning and reinforcement learning is changing the landscape and emerging as the winner for optimal frame selection. Recent advances in GANs are improving the quality, aesthetics and relevancy of the frames to represent the original videos. Come join this session to get an understanding of various challenges and emerging solutions around video summarization.

Divya Jain at AI Frontiers : Video Summarization

AI Frontiers

Topic: How to use big data to enhance AI Outline: 1. Spark ETL Spark SQL Spark Streaming 2. Spark ML Spark ML pipeline Distributed model tuning Spark ML model and data lineage management 3. Spark XGboost XGboost introduction XGboost with Spark XGboost with GPU 4. Spark Deep Learning pipeline Transfer learning Build Spark ML pipeline with TensorFlow Model selection on distributed TF model

Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI

AI Frontiers

Topic：Heuristic Search and how does it apply to self-driving cars? (for Beginners) Outline: 1. Technologies behind self-driving vehicles - Motion Planning - Decision Making 2. Graph Search Algorithms - Depth-First Search - Breadth-First Search - A* Search 3. Incremental Heuristic Search Algorithms - Repeated A* Search - Adaptive A* - Generalized Fringe-Retrieving A*

Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 1: Heuristi...

AI Frontiers

In this tutorial I will introduce recent work in applying weak supervision and reinforcement learning to Questions Answering (QA) systems. Specifically we discuss the semantic parsing task for which natural language queries are converted to computation steps on knowledge graphs or data tables and produce the expected answers. State-of-the-art results can be achieved by novel memory structure for sequence models and improvements in reinforcement learning algorithms. Related code and experiment setup can be found at https://github.com/crazydonkey200/neural-symbolic-machines. Related paper: https://openreview.net/pdf?id=SyK00v5xx.

Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...

AI Frontiers

Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-lecture 2: Incremen...

AI Frontiers

Instructor: Mat Leonard Outline 1. Text Processing Using Python + NLTK Cleaning Normalization Tokenization Part-of-speech Tagging Stemming and Lemmatization 2. Feature Extraction Bag of Words TF-IDF Word Embeddings Word2Vec GloVe 3. Topic Modeling Latent Variables Beta and Dirichlet Distributions Laten Dirichlet Allocation 4. NLP with Deep Learning Neural Networks Recurrent Neural Networks (RNNs) Word Embeddings Sentiment Analysis with RNNs

Training at AI Frontiers 2018 - Udacity: Enhancing NLP with Deep Neural Networks

AI Frontiers

Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 3: Any-Angl...

AI Frontiers

Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...

AI Frontiers

In recent years, machine learning has undoubtedly been hugely successful in driving progress in AI applications. However, as we will explore in this talk, even state-of-the-art systems have "blind spots" which make them generalize poorly out of domain and render them vulnerable to adversarial examples. We then suggest that more unsupervised learning settings can encourage the development of more robust systems. We show positive results on two tasks: (i) text style and attribute transfer, the task of converting a sentence with one attribute (e.g., sentiment) to one with another; and (ii) solving SAT instances (classical problems requiring logical reasoning) using end-to-end neural networks.

Percy Liang at AI Frontiers : Pushing the Limits of Machine Learning

AI Frontiers

I will present several advances in deep learning from OpenAI. First, I will present OpenAI Five, a neural network that learned to play on par with some of the strongest professional Dota 2 teams in the world in an 18-hero version of the game. Next, I will present Dactyl, a human-like robot hand trained entirely in simulation with reinforcement learning that has achieved unprecedented dexterity on a physical robot. I will also present our results on unsupervised learning in language, that show that pre-training and finetuning can achieve a significant improvement over state of the art. Finally, I will present an overview of the historical progress in the field.

Ilya Sutskever at AI Frontiers : Progress towards the OpenAI mission

AI Frontiers

Mark Moore at AI Frontiers : Uber Elevate

AI Frontiers

Arnaud Thiercelin at AI Frontiers : AI in the Sky

AI Frontiers

As the data and models scale, it becomes necessary to have multiple processing units for both training and inference. SignSGD is a gradient compression algorithm that only transmits the sign of the stochastic gradients during distributed training. This algorithm uses 32 times less communication per iteration than distributed SGD. We show that signSGD obtains free lunch both in theory and practice: no loss in accuracy while yielding speedups. Pushing the current boundaries of deep learning also requires using multiple dimensions and modalities. These can be encoded into tensors, which are natural extensions of matrices. These functionalities are available in the Tensorly package with multiple backend interfaces for large-scale deep learning.

Anima Anandkumar at AI Frontiers : Modern ML : Deep, distributed, Multi-dimen...

AI Frontiers

Wei Xu at AI Frontiers : Language Learning in an Interactive and Embodied Set...

AI Frontiers

The use of AI for voice search and image recognition is talked about often. Enterprises, however, have different challenges and requirements. In this talk, we will focus on talking about use cases in the enterprise and challenges in building out AI solutions. We will talk about how an Auto-machine learning software for videos and images called PowerAI Vision enables quick AI model training & deployment for various enterprise use cases.

Sumit Gupta at AI Frontiers : AI for Enterprise

AI Frontiers

Deep Reinforcement Learning (DRL) has made strong progress in many tasks, such as board games, robotics, navigation, neural architecture search, etc. I will present our recent open-sourced DRL frameworks to facilitate game research and development. Our framework is scalable so we can can reproduce AlphaGoZero and AlphaZero using 2000 GPUs, achieving super-human performance of Go AI that beats 4 top-30 professional players. We also show usability of our platform by training agents in real-time strategy games, and show interesting behaviors with a small amount of resource.

Yuandong Tian at AI Frontiers : Planning in Reinforcement Learning

AI Frontiers

The latest AI advances have the potential to massively improve our health and well being. However, most of the work is yet to be done. In this talk, we will explore the most important opportunities for AI in healthcare. For example, we will explore how AI can diagnose major life-threatening conditions even before those conditions emerge. We will talk about AI ability to recommend dramatically more effective and less harmful treatment plans based on AI understanding of patient's medical history and current conditions. Finally, we will talk about AI role in making our healthcare system effective and affordable for everyone.

Alex Ermolaev at AI Frontiers : Major Applications of AI in Healthcare

AI Frontiers

Games have been leveraging AI since the 1950s, when people built a rules-based AI engine that played tic-tac-toe. With technological advances over the years, AI has become increasingly popular and widely used in the gaming industry. The typical characteristics of games and game development makes them an ideal playground for practicing and implementing AI techniques, especially deep learning and reinforcement learning. Most games are well scoped; it is relatively easy to generate and use the data; and states/actions/rewards are relatively clear. In this talk, I will show a couple of use cases where ML/AI helps in-game development and enhances player experience. Examples include AI agents playing game and services that provide personalized experience to players.

Long Lin at AI Frontiers : AI in Gaming

AI Frontiers

Melissa Goldman at AI Frontiers : AI & Finance

AI Frontiers

Li Deng at AI Frontiers : From Modeling Speech/Language to Modeling Financial...

AI Frontiers

More from AI Frontiers (20)