Challenges of managing Data Science Project

•

2 j'aime•779 vues

This document discusses challenges with a data science project including the experimental and unpredictable nature, difficulties with scheduling due to unknown completion times, proposing new products with data and technology limitations, and addressing failures by breaking the work into smaller pieces with increased visibility and producing functional interim results to stay on track.

Données & analyses

Ambiguous and unpredictable!
What to do?
Do you find the best idea?
Do you have good results?

Scheduling almost impossible !
When you’ll be done?

Propose a new product?
Limitation of data and technology?

Break down into smallpieces
Increase visibility!

Makeit work!
Produceasomething thatfunctionsenough tokeep youontrack

Acknowledgefailure
Figure out what to do next!

Contenu connexe

Tendances

How To Stop Procrastinating

drededmegz247

Big data, Big Results

Steven Stegman

Lucidchart has been recognized as one of the most mature Product Lead Growth business models in the market, driving over 700,000 registrations per month, combined with a hyper-efficient B2B business model. It didn’t happen by accident. It happened through experimentation: from extensive A/B testing (with over 500 marketing tests completed in 2018), to testing crazy brand videos (which have garnered over 200 million views and was named the only ad campaign that truly mattered in 2018 by Adweek), to constant iteration and expansion of the business model. Experimental marketing brings together the science and art of marketing, allowing for creativity that drives results. It’s an essential skill in the toolkit of the modern marketer, and one that’s easy to get started with, no matter your company scale. Key Takeaways • Discover how to start or extend your marketing experimentation. • Learn how marketing experimentation can apply across funnels, brands, and business models. • Get tips on how to build a culture of experimentation that fosters creativity and drives business results.

Top Lessons Learned From Our Best and Worst Marketing Experiments with Lucidc...

saastr

GTD(R) Workshop

Kevin Davison

Wealth Mindset- Best practices to get Organized!

MontySalt

Mark Richardson is highly regarded as a pioneer in developing standards of professionalism in the construction industry. Join Mark as he reviews his new book and how roofing and remodeling professionals can improve their time management. Highlights: - Why it’s important to understand time today - The differences in being efficient & effective, and proactive vs. reactive time - How to establish a time mastery system that allows you to take control of your day, so your day doesn’t control you - Time-tested tactics that help you avoid common mistakes that cost you time

Mark G. Richardson Presents: Control Your Day Before Your Day Controls You

Surefire Local

How To Stop Procrastinating

drededmegz247

12 Powerful Time Management Tips that will let you work less and achieve more.

Vartika Kashyap

The art of shifting perspectives - Rachel Davies

JAXLondon_Conference

Design Cyle Lesson 2

Tonia.Johnson

12 Months of Failure: Lessons Learned in Year One of Bromford Lab

Bromford Lab

Slides Andy Wadhwa recently used in his discussion w/ mentees of The Product Mentor. Synopsis: Getting things done and accomplishing more in less time is an especially important skill for product managers. During this talk, Andy will go over a list of recommendations he has found to be helpful in boosting his level of productivity. The Product Mentor is a program designed to pair Product Mentors and Mentees from around the World, across all industries, from start-up to enterprise, guided by the fundamental goals…Better Decisions. Better Products. Better Product People. Throughout the program, each mentor leads a conversation in an area of their expertise that is live streamed and available to both mentee and the broader product community. http://TheProductMentor.com

Productivity Hacks for Product Managers

Jeremy Horn

Big Data Lab for the Canadian government

Jason Annable

Working Hard or Hardly Working: Simple Office Design Tips

San Diego Office

Agile mindset

Dhanuka Senaratne

Managing work flow- ppt

Nandu Warrier

Lean start up & innovation

Elo Umeh

Zen and Art of Workflow Development

NTEN

When you are feeling overloaded, instead of pushing harder, a wise thing to do might be to take a short break. As we discussed in one of our previous posts, short breaks can revive your productivity and decrease tension. We looked deeper into the matter and put together some interesting facts, as well as practical advice on how to stay fresh and productive throughout the day. Here’s a guide to productive breaks in our new infographic! www.wrike.com/

The science of productive breaks

Wrike

10 things I've learnt about lean startup: It's f*****g hard that's for sure!

The Happy Startup School

Tendances (20)

How To Stop Procrastinating

Big data, Big Results

Top Lessons Learned From Our Best and Worst Marketing Experiments with Lucidc...

GTD(R) Workshop

Wealth Mindset- Best practices to get Organized!

Mark G. Richardson Presents: Control Your Day Before Your Day Controls You

How To Stop Procrastinating

12 Powerful Time Management Tips that will let you work less and achieve more.

The art of shifting perspectives - Rachel Davies

Design Cyle Lesson 2

12 Months of Failure: Lessons Learned in Year One of Bromford Lab

Productivity Hacks for Product Managers

Big Data Lab for the Canadian government

Working Hard or Hardly Working: Simple Office Design Tips

Agile mindset

Managing work flow- ppt

Lean start up & innovation

Zen and Art of Workflow Development

The science of productive breaks

10 things I've learnt about lean startup: It's f*****g hard that's for sure!

En vedette

The Other 99% of a Data Science Project

Eugene Mandel

We describe in this paper our participation in the product search task of LL4IR CLEF 2015 Lab. This task aims to evaluate, with living labs protective point of view, the retrieval effectiveness over e-commerce search engines. During the online shopping process, users would search for interesting products and quickly access those that fit with their needs among a long tail of similar or closely related products. Our contribution addresses head queries that are frequently submitted on e-commerce Web sites. Head queries usually target featured products with several variations, accessories, and complementary products. We propose a probabilistic model for product search based on the intuition that descriptive fields and the category might fit with the query. Finaly, we present results obtained during the second round of the product search task.

IRIT at clef 2015: A product search model for head queries

Lamjed Ben Jabeur

The research topic of this document deals with a particular setting of information retrieval (IR), referred to as collaborative information retrieval (CIR), in which a set of multiple collaborators share the same information need. Collaboration is particularly used in case of complex tasks in which an individual user may have insufficient knowledge and may benefit from the expertise/knowledge or complementarity of other collaborators. This multi-user context rises several challenges in terms of search interfaces as well as ranking models, since new paradigms must be considered, namely division of labor, sharing of knowledge and awareness. These paradigms aim at avoiding redundancy between collaborators in order to reach a synergic effect within the collaboration process. Several approaches have been proposed in the literature. First, search interfaces have been oriented towards a user mediation in order to support collaborators’ actions through information storage or communication tools. Second, more close to our contributions, previous work focus on the information access issue by designing ranking models adapted to collaborative environments dealing with the challenges of (1) personalizing result set to collaborators, (2) favoring the sharing of knowledge, (3) dividing the labor among collaborators and/or (4) considering particular roles of collaborators within the information seeking process. In this thesis, we focus, more particularly, on two main aspects of the collaboration : — The expertise of collaborators by proposing retrieval models adapted to the domain expertise level of collaborators. The expertise levels might be vertical, in the case of domain expert and novice, or horizontal when collaborators have different subdomain expertise. We, therefore, propose two CIR models on two steps including a document relevance scoring with respect to each role and a document allocation to user roles through the Expectation–Maximization (EM) learning method applied on the document relevance scoring in order to assign documents to the most likely suited user. — The complementarity of collaborators throughout the information seeking process by mining their roles on the assumptions that collaborators might be different and complementary in some skills. We propose two algorithms based either on predefined roles or latent roles which (1) learns about the roles of the collaborators using various search-related features for each individual involved in the search session, and (2) adapts the document ranking to the mined roles of collaborators.

Thesis slides - Definition and evluation of collaborative information retriev...

UPMC - Sorbonne Universities

Information Retrieval UMLS

siddarajuss

Information Retrieval

Phuong Pham

Unified Medical Language System & MetaMap

Osama Jomaa

Description and retrieval of medical visual information based on language mod...

Antonio Foncubierta Rodriguez

Health IT & Voice of Patient

Dr.Mahmoud Abbas

2. Triết học MBA (LS triết học Ấn Độ)

Hưng, Đinh Duy

Medical Information Retrieval and its Evaluation: an Overview of CLEF eHealth...

lorrainegoeuriot

Five awesome django tutorials - Open Data Science

opendatascience

Despite a wide array of advanced techniques available today, too many practitioners are forced to return to their old toolkit of approaches deemed “more interpretable.” Whether because of non-legal policy or difficulty in executive presentation, these restraints result from poor analytics communication and inability to explain model risks and outcomes, not a failing of the techniques. From sampling to feature reduction to supervised modeling, the toolbox and communications of data scientists are limited by these constraints. But, instead of simplifying models, data scientists can re-introduce often ignored statistical practices to describe the models, their risk, and the impact of changes in the customer environment. Even in situations without restrictions, these approaches will improve how practitioners select models and communicate results. Through measurement and simulation, reviewed approaches can be used to articulate the promises, risks, and assumptions of developed models, without requiring deep statistical explanations.

Dan Mallinger, Data Science Practice Manager, Think Big Analytics at MLconf NYC

MLconf

Enterprise companies starting the transformation into a data-driven organization often wonder where to start. Companies have traditionally collected large amounts of data from sources such as operational systems. With the rise of big data, big data technologies and the Internet of Things (IoT), additional sources – such as sensor readings and social media posts – are rapidly becoming available. In order to effectively utilize both traditional sources and new ones, companies first need to join and view the data in a holistic context. After establishing a data lake to bring all data sources together in a single analytics environment, one of the first data science projects worth exploring is segmentation, which automatically identifies patterns. In this DSC webinar, two Pivotal data scientists will discuss: · What segmentation is · Traditional approaches to segmentation · How big data technologies are enabling advances in this field They will also share some stories from past data science engagements, outline best practices and discuss the kinds of insights that can be derived from a big data approach to segmentation using both internal and external data sources. Panelist: Grace Gee, Data Scientist -- Pivotal Jarrod Vawdrey, Data Scientist -- Pivotal Hosted by: Tim Matteson, Co-Founder -- Data Science Central To learn more about data at Pivotal, visit http://www.pivotal.io/big-data To view video, visit https://www.youtube.com/watch?v=svKLdMWusGA

Webinar - The Science of Segmentation: What Questions You Should be Asking Yo...

VMware Tanzu

This talk will introduce a paradigm for enabling access to large, unstructured, and novel datasets in enterprises, while retaining value from existing tools and staff. By following a real world example, the discussion will walk through how small, central data science teams can make data discoveries and data value accessible to others. We will also review the tools, data science approaches, and best practices to uncovering, polishing, and digesting signal in data to support analytics at the front lines of business.

Dan Mallinger – Data Science Practice Manager, Think Big Analytics at MLconf ATL

MLconf

Laure Soulier, Lamjed Ben Jabeur, Paul Mousset, Lynda Tamine. Quels facteurs de pertinence pour la recherche de produits e-commerce ?. Dans : Conférence francophone en Recherche d'Information et Applications (CORIA 2016), Toulouse, 09/03/2016-11/03/2016, Association Francophone de Recherche d'Information et Applications (ARIA), p. 415-430, mars 2016. https://www.irit.fr/publis/SIG/2016_CORIA_SOULIER.pdf Un moteur de recherche e-commerce vise à fournir un accès rapide et efficace à des produits qui correspondent aux besoins et aux préférences de l'utilisateur parmi une liste de produits similaires ou étroitement liés. Nous avons participé à la campagne d'évaluation « Living Lab for Information Retrieval » qui proposait une tâche de recherche de produits évaluée par des utilisateurs réels lors de scénarios de recherche réelle sur un site de e-commerce. L’évaluation expérimentale a montré des résultats prometteurs de notre modèle. Dans ce papier, nous proposons une analyse des fichiers logs issus de notre modèle afin d'identifier des facteurs d’efficacité liés à la requête et aux produits. L'objectif de cette étude est d'ouvrir des pistes de recherche pour la formalisation de modèles de recherche de produits. E-commerce product retrieval aims to provide a quick and efficient access to products that fit user’s needs and preferences among a tail of similar or closely related products. We participated to the ``Living Lab for Information Retrieval'' evaluation campaign devoted to a product search task in which real users evaluated participants' retrieval models in real search scenarios on e-commerce websites. The experimental evaluation has shown encouraging results for our proposed model. In this paper, we conduct an analysis of users' feeadback with respect to the clicks obtained by our model. The goal of the paper is therefore to identify the effectiveness factors underlying the user's queries and the retrieved products in order to open perspectives in the formalization of product search models.

Quels facteurs de pertinence pour la recherche de produits e-commerce ?

Lamjed Ben Jabeur

Idiots guide to setting up a data science team

Ashish Bansal

CRISP-DM - Agile Approach To Data Mining Projects

Michał Łopuszyński

CRISP-DM: a data science project methodology

Sergey Shelpuk

Generally speaking, big data and data science originated in the west and are coming to Europe with a bit of a delay. There is at least one exception though: The London-based music discovery website Last.fm is a data company at heart and has been doing large-scale data processing and analysis for years. It started using Hadoop in early 2006, for instance, making it one of the earliest adopters worldwide. When I left Last.fm to join Massive Media, the social media company behind Netlog.com and Twoo.com, I basically moved from a data science forerunner to a newcomer. Massive Media had at least as much data to play with and tremendous potential, but they were not doing much with it yet. The data science team had to be build from the ground up and every step had to be argued for and justified along the way. Having done this exercise of evaluating everything I learned at Last.fm and starting over completely with a clean slate at Massive Media, I developed a pretty clear perspective on how to find good data scientists, what they should be doing, what tools they should be using, and how to organize them to work together efficiently as team, which is precisely what I would like to share in this talk.

Back to Square One: Building a Data Science Team from Scratch

Klaas Bosteels

Organizations need to perform increasingly complex analysis on their data — streaming analytics, ad-hoc querying and predictive analytics — in order to get better customer insights and actionable business intelligence. However, the growing data volume, speed, and complexity of diverse data formats make current tools inadequate or difficult to use. Apache Spark has recently emerged as the framework of choice to address these challenges. Spark is a general-purpose processing framework that follows a DAG model and also provides high-level APIs, making it more flexible and easier to use than MapReduce. Thanks to its use of in-memory datasets (RDDs), embedded libraries, fault-tolerance, and support for a variety of programming languages, Apache Spark enables developers to implement and scale far more complex big data use cases, including real-time data processing, interactive querying, graph computations and predictive analytics. In this session, we present a technical deep dive on Spark running on Amazon EMR. You learn why Spark is great for ad-hoc interactive analysis and real-time stream processing, how to deploy and tune scalable clusters running Spark on Amazon EMR, how to use EMRFS with Spark to query data directly in Amazon S3, and best practices and patterns for Spark on Amazon EMR.

(BDT309) Data Science & Best Practices for Apache Spark on Amazon EMR

Amazon Web Services

En vedette (20)

The Other 99% of a Data Science Project

IRIT at clef 2015: A product search model for head queries

Thesis slides - Definition and evluation of collaborative information retriev...

Information Retrieval UMLS

Information Retrieval

Unified Medical Language System & MetaMap

Description and retrieval of medical visual information based on language mod...

Health IT & Voice of Patient

2. Triết học MBA (LS triết học Ấn Độ)

Medical Information Retrieval and its Evaluation: an Overview of CLEF eHealth...

Five awesome django tutorials - Open Data Science

Dan Mallinger, Data Science Practice Manager, Think Big Analytics at MLconf NYC

Webinar - The Science of Segmentation: What Questions You Should be Asking Yo...

Dan Mallinger – Data Science Practice Manager, Think Big Analytics at MLconf ATL

Quels facteurs de pertinence pour la recherche de produits e-commerce ?

Idiots guide to setting up a data science team

CRISP-DM - Agile Approach To Data Mining Projects

CRISP-DM: a data science project methodology

Back to Square One: Building a Data Science Team from Scratch

(BDT309) Data Science & Best Practices for Apache Spark on Amazon EMR

Plus de Lamjed Ben Jabeur

Nous présentons dans cet article une analyse exploratoire dans l’objectif d’identifier les patrons de collaboration au sein des réseaux sociaux pour l’accès à l’information. Nous nous focalisons sur le cas de la gestion de crise au travers de la plateforme sociale Twitter. Nous nous appuyons ainsi sur deux types de crises : une catastrophe naturelle (l’ouragan Sandy) et une crise sanitaire (Ebola). Nous proposons une méthodologie en trois étapes qui permet 1) d’analyser les structures de collaboration, 2) d’identifier les thématiques de collaboration et 3) de présenter les réseaux latents de collaboration sur l’ensemble du réseau. Les résultats montrent paradoxalement que le périmètre de la collaboration est réduit, que les sujets sousjacents sont diversifiés et impliquent des utilisateurs qui peuvent être à l’interface de différents sujets.

Accès à l’information dans les réseaux sociaux : quelles formes de collaborat...

Lamjed Ben Jabeur

Leveraging social relevance: Using social networks to enhance literature access and microblog search (Exploitation des réseaux sociaux pour l'accès à la littérature et la recherche des microblogs) Thesis submitted for the degree of Doctor of Philosophy Thesis defended on October 8th, 2013 Ph.D: Lamjed Ben Jabeur Supervisor: Prof. Lynda Tamine, University of Toulouse 3 Paul Sabatier Advisor: Prof. Mohand Boughanem, University of Toulouse 3 Paul Sabatier Abstract(EN) We propose in this work to integrate the social information network in the retrieval process and exploit the social relations between social actors as a source of evidence to measure the relevance of a document in response to a query. Two social information retrieval models have been proposed in different application frameworks: literature access and microblog retrieval. The main contributions of each model are detailed in the following. * A social information model for flexible literature access We proposed a generic social information retrieval model for literature access. This model represents scientific papers within a social network and evaluates their importance according to the position of respective authors in the network. Compared to previous approaches, this model incorporates new social entities represented by annotators and social annotations (tags). In addition to co-authorships, this model includes two other types of social relationships: citation and social annotation. Finally, we propose to weight these relationships according to the position of authors in the social network and their mutual collaborations. * A social model for information retrieval for microblog search We proposed a microblog retrieval model that evaluates the quality of tweets in two contexts: the social context and temporal context. The quality of a tweet is estimated by the social importance of the corresponding blogger. In particular, blogger's importance is calculated by the applying PageRank algorithm on the network of social influence. With the same aim, the quality of a tweet is evaluated according to its date of publication. Tweets submitted in periods of activity of query terms are then characterized by a greater importance. Finally, we propose to integrate the social importance of blogger and the temporal magnitude tweets as well as other relevance factors using a Bayesian network model. Résumé (FR) Nous proposons dans cette thèse d'intégrer le réseau social d'information dans le processus de recherche d'information afin d'utiliser les relations sociales entre les acteurs sociaux comme une source d'évidence pour mesurer la pertinence d'un document en réponse à une requête. Deux modèles de recherche d'information sociale ont été proposés à des cadres applicatifs différents : la recherche d'information bibliographique et la recherche d'information dans les microblogs.

Leveraging social relevance: Using social networks to enhance literature acce...

Lamjed Ben Jabeur

Lamjed Ben Jabeur, Lynda Tamine, Mohand Boughanem. Uprising microblogs: A Bayesian network retrieval model for tweet search (regular paper). Dans : ACM Symposium on Applied Computing (SAC 2012), Riva del Garda (Trento), Italy, 26/03/12-30/03/12, mars 2012 http://dx.doi.org/10.1145/2245276.2245459 We investigate in this paper the problem of accessing to real-time information and we propose a Bayesian network retrieval model for tweet search. The proposed model interprets tweet relevance as a conditional probability and estimates it using different sources of evidence. In particular, we introduce a social search model that considers, in addition to text similarity measures, the microblogger’s influence, the time magnitude and the presence of hashtags. To evaluate our model, we conducted a series of experiments on the TREC Tweets2011 corpus. Experiments with “Arab Spring” topic set show that both of social and temporal features improve tweet search for different types of queries. Final results show also that our model outperforms other traditional information retrieval baselines.

Uprising microblogs: A Bayesian network retrieval model for tweet search

Lamjed Ben Jabeur

Lamjed Ben Jabeur, Lynda Tamine, Mohand Boughanem. Intégration des facteurs temps et autorité sociale dans un modèle bayésien de recherche de tweets. Dans : Conférence francophone en Recherche d'Information et Applications (CORIA 2012), Bordeaux, 21/03/12-23/03/12, LABRI, p. 301-316, 2012 ftp://ftp.irit.fr/IRIT/SIG/BenJabeur_CORIA2012.pdf Cet article présente une approche sociale pour la recherche d’information dans les microblogs intégrant diverses sources d’évidence au sein d’un réseau bayésien. Notre contribution consiste à étendre la notion classique de pertinence, basée sur la similarité textuelle, par de nouveaux facteurs tels que l’importance sociale des blogueurs et la magnitude temporelle des microblogs. Dans ce papier, l’importance sociale d’un blogueur est assimilée à son influence dans le réseau et est évaluée par un score de PageRank déduit sur le réseau de diffusion des microblogs. Nous proposons d’estimer la magnitude temporelle selon le nombre de voisins temporels qui incluent les termes de la requête. Afin de valider notre approche, une évaluation expérimentale à été menée sur la collection de microblogs de référence TREC Tweets2011. Les résultats montrent que notre modèle présente un gain de 24% par rapport à la médiane des résultats officiels de TREC Microblog 2011.

Intégration des facteurs temps et autorité sociale dans un modèle bayésien de...

Lamjed Ben Jabeur

Un modèle de recherche d’information sociale dans les microblogs : cas de Twi...

Lamjed Ben Jabeur

Cet article propose une nouvelle approche, basée sur les réseaux sociaux, pour l'accès aux ressources bibliographiques. Nous introduisons un modèle d'information sociale dont les auteurs sont les principales entités et les relations sont extraites à partir des liens de coauteur et de citation. En effet, ces relations sont pondérées en tenant compte des interactions entre les auteurs et des annotations sociales produites par les utilisateurs. Dans ce modèle, la pertinence d'un document est estimée par combinaison de la pertinence thématique et de la pertinence sociale, qui est à son tour dérivée de l'importance sociale des auteurs associés. Nous évaluons la viabilité de notre modèle sur une collection d'articles scientifiques dont les annotation sociales sont extraites depuis le réseau social académique CiteULike.org. Les résultats obtenus montrent la supériorité des performances de notre modèle par rapport à la recherche d'information traditionnelle.

Un modèle de Recherche d'Information Sociale pour l'Accès aux Ressources Bib...

Lamjed Ben Jabeur

This paper presents a novel retrieval approach for literature access based on social network analysis. In fact, we investigate a social model where authors represent the main entities and relationships are extracted from co-author and citation links. Moreover, we define a weighting model for social relationships which takes into account the authors positions in the social network and their mutual collaborations. Assigned weights express influence, knowledge transfer and shared interest between authors. Furthermore, we estimate document relevance by combing the document-query similarity and the document social importance derived from corresponding authors. To evaluate the effectiveness of our model, we conduct a series of experiments on a scientific document dataset that includes textual content and social data extracted from the academic social network "CiteULike". Final results show that the proposed model improves the retrieval effectiveness and outperforms traditional and social information retrieval baselines.

A social model for Literature Access: Towards a weighted social network of au...

Lamjed Ben Jabeur

It is well known that the fundamental intellectual problems of information access are the production and consumption of information. In this paper, we investigate the use of social network of information producers (authors) within relations in data (co-authorship and citation) in order to improve the relevance of information access. Relevance is derived from the network by levraging the usual topical similarity between the query and the document with the target author’s authority. We explore various social network based measures for computing social information importance and show how this kind of contextual information can be incorporated within an information access model. We experiment with a collection issued from SIGIR proceedings and show that combining topical, author and citation based evidences can significantly improve retrieval access precision, measured in terms of mean reciprocal rank.

An Exploratory Study on Using Social Information Networks for Flexible Litera...

Lamjed Ben Jabeur

Plus de Lamjed Ben Jabeur (8)