Deep Learning in Deep Space

•

4 j'aime•4,088 vues

This document discusses the use of machine learning techniques for analyzing astronomical and earth observation data. It notes that large sky survey projects generate huge amounts of data that exceeds our ability to analyze using traditional methods. Machine learning can help process and extract insights from this big data. Specifically, the document discusses how convolutional neural networks have achieved 98% accuracy in classifying galaxy images from sky surveys. It also provides examples of applying machine learning to tasks like terrain classification from earth observation satellite imagery.

Données & analyses

Deep Learning in Deep Space
Applica(ons of Machine Learning
on Astronomical Data
Dr. Brian Mac Namee
brian.macnamee@ucd.ie
@brianmacnamee

Big sky survey projects are amongst the biuggest data sources available
Sloan Digital Sky Survey (www.sdss.org)

The SDSS telescopes
collect over 200GB of
data every night

Over 200 million
galaxies have been
photographed

Data has become a
huge boPle neck in
modern astronomy

Future telescopes will
collect even more
data
Sloan Digital Sky Survey (www.sdss.org)

Data
Collec(on
Data
Processing
Science
Machine
Learning
* Connec(on from data to Brahe to Newton
stolen from Prof. James Gleeson (UL)

Data
Collec(on
Data
Processing
Science
Machine
Learning
* Connec(on from data to Brahe to Newton
stolen from Prof. James Gleeson (UL)
(Machine Learning)
Correla(on
or
Causa(on
(Sta(s(cs)

Ellipse
Le0 Spiral
Right Spiral
Fundamentals of Machine Learning
for Predic(ve Data Analy(cs
John D. Kelleher, Brian Mac Namee, Aoife D'Arcy
MIT Press
www.machinelearningbook.com

Support Vector Machine Performance
Fundamentals of Machine Learning for Predic(ve Data Analy(cs
John D. Kelleher, Brian Mac Namee, Aoife D'Arcy
MIT Press
www.machinelearningbook.com
Overall accuracy:
~88%

Galaxy Catalogue (hPp://www.zsolt-frei.net/galaxy_catalog.html)
Data is the Fuel of Machine Learning

Galaxy Catalogue (hPp://www.zsolt-frei.net/galaxy_catalog.html)
Data is the Fuel of Machine Learning
Labelled data
is the real key

Galaxy Zoo (www.galaxyzoo.org)
Crowdsourcing Labelled Data

Galaxy Zoo (www.galaxyzoo.org)
Crowdsourcing Labelled Data
Almost 1,000,000 galaxies
labelled

Over 150,000 users

Millions of individual
labels

Other uses of machine learning in astronomy:
– Object classiﬁca(on (stars, galaxies, quasars, ...)
– Es(ma(ng redshil
– Es(ma(ng other parameters of sky objects
– Forecas(ng solar ﬂare ac(vity
Kremer, Jan, Kristoﬀer Stensbo-Smidt, Fabian Gieseke, Kim Steenstrup Pedersen, and Chris(an Igel. "Big Universe, Big Data: Machine Learning
and Image Analysis for Astronomy." IEEE Intelligent Systems 32, no. 2 (2017): 16-22.
Ball, Nicholas M., and Robert J. Brunner. "Data mining and machine learning in astronomy." Interna)onal Journal of Modern Physics D 19, no. 07
(2010): 1049-1106.

European Space Agency (www.galaxyzoo.org)
Earth Observa(on

Earth Observa(on Terrain
classiﬁca(on is a
standard task

Convolu(onal
neural networks
are leading
approach for this
task

Accuracies of
>90% possible
Classiﬁca(on and segmenta(on of
satellite orthoimagery using
convolu(onal neural networks,
Längkvist, Mar(n, Andrey Kiselev,
Marjan Alirezaie, and Amy Lousi,
Remote Sensing 8, no. 4 (2016): 329.

Other uses of machine learning in earth
observa(on:
– Oﬀers poten(al to use mul(-spectral images
– Terrain classiﬁca(on
– Target recogni(on
– Seman(c feature extrac(on
– Scene understanding

Thank you

Ques<ons?
Dr. Brian Mac Namee
brian.macnamee@ucd.ie
@brianmacnamee
www.bigskyearth.eu
www.machinelearningbook.com

Contenu connexe

Tendances

Detecting solar farms with deep learning

Jason Brown

OCC Overview OMG Clouds Meeting 07-13-09 v3

Robert Grossman

Accelerating Science with Cloud Technologies in the ABoVE Science Cloud

Globus

Using parallel hierarchical clustering to

Biniam Behailu

2014 moore-ddd

c.titus.brown

TERN eMAST : Observations and terrestrial ecosystem models : Terrestrial Ecos...

Brad Evans

Open Science Data Cloud - CCA 11

Robert Grossman

Open Science Data Cloud (IEEE Cloud 2011)

Robert Grossman

Application of web ontology to harvest estimation of rice in thailand

AIMS (Agricultural Information Management Standards)

Application of web ontology to harvest estimation of rice in Thailand

AIMS (Agricultural Information Management Standards)

LSST Solar System Science: MOPS Status, the Science, and Your Questions

Mario Juric

educ1751 - analysis of a technology mediated enquiry

debbie evans

Health & Status Monitoring (2010-v8)

Robert Grossman

A Biological Internet?: Eywa

Eugene Siow

Computational Training and Data Literacy for Domain Scientists

Joshua Bloom

Bionimbus Cambridge Workshop (3-28-11, v7)

Robert Grossman

Many Task Applications for Grids and Supercomputers

Ian Foster

Denis Reznik Data driven future

Аліна Шепшелей

An Overview of Bionimbus (March 2010)

Robert Grossman

In this deck from the 2017 Argonne Training Program on Extreme-Scale Computing, Rupak Biswas from NASA presents: NASA Advanced Computing Environment for Science & Engineering. ""High performance computing is now integral to NASA’s portfolio of missions to pioneer the future of space exploration, accelerate scientific discovery, and enable aeronautics research. Anchored by the Pleiades supercomputer at NASA Ames Research Center, the High End Computing Capability (HECC) Project provides a fully integrated environment to satisfy NASA’s diverse modeling, simulation, and analysis needs. In addition, HECC serves as the agency’s expert source for evaluating emerging HPC technologies and maturing the most appropriate ones into the production environment. This includes investigating advanced IT technologies such as accelerators, cloud computing, collaborative environments, big data analytics, and adiabatic quantum computing. The overall goal is to provide a consolidated bleeding-edge environment to support NASA's computational and analysis requirements for science and engineering applications." Dr. Rupak Biswas is currently the Director of Exploration Technology at NASA Ames Research Center, Moffett Field, Calif., and has held this Senior Executive Service (SES) position since January 2016. In this role, he in charge of planning, directing, and coordinating the technology development and operational activities of the organization that comprises of advanced supercomputing, human systems integration, intelligent systems, and entry systems technology. The directorate consists of approximately 700 employees with an annual budget of $160 million, and includes two of NASA’s critical and consolidated infrastructures: arc jet testing facility and supercomputing facility. He is also the Manager of the NASA-wide High End Computing Capability Project that provides a full range of advanced computational resources and services to numerous programs across the agency. In addition, he leads the emerging quantum computing effort for NASA. Watch the video: https://wp.me/p3RLHQ-hua Learn more: https://extremecomputingtraining.anl.gov/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

NASA Advanced Computing Environment for Science & Engineering

inside-BigData.com

Tendances (20)

Detecting solar farms with deep learning

OCC Overview OMG Clouds Meeting 07-13-09 v3

Accelerating Science with Cloud Technologies in the ABoVE Science Cloud

Using parallel hierarchical clustering to

2014 moore-ddd

TERN eMAST : Observations and terrestrial ecosystem models : Terrestrial Ecos...

Open Science Data Cloud - CCA 11

Open Science Data Cloud (IEEE Cloud 2011)

Application of web ontology to harvest estimation of rice in thailand

Application of web ontology to harvest estimation of rice in Thailand

LSST Solar System Science: MOPS Status, the Science, and Your Questions

educ1751 - analysis of a technology mediated enquiry

Health & Status Monitoring (2010-v8)

A Biological Internet?: Eywa

Computational Training and Data Literacy for Domain Scientists

Bionimbus Cambridge Workshop (3-28-11, v7)

Many Task Applications for Grids and Supercomputers

Denis Reznik Data driven future

An Overview of Bionimbus (March 2010)

NASA Advanced Computing Environment for Science & Engineering

Similaire à Deep Learning in Deep Space

A 100 gigabit highway for science: researchers take a 'test drive' on ani tes...

balmanme

Efficient data reduction and analysis of DECam images using multicore archite...

Roberto Muñoz

Data Science at Berkeley

Joshua Bloom

Describes data science efforts at Berkeley, with a particular focus on teaching and the new Berkeley Institute for Data Science (BIDS), funded by the Moore and Sloan Foundations. BIDS will be a space for the open and interdisciplinary work that is typical of the SciPy community. In the creation of BIDS, open source scientific tools for data science, and specifically the SciPy ecosystem, played an important role.

Joshua Bloom Data Science at Berkeley

PyData

Agents In An Exponential World Foster

Ian Foster

Surfing the Exponential for Forty Years

Larry Smarr

Presentation

farrelle25

Keynote on 2015 Yale Day of Data

Robert Grossman

Modern astronomical surveys are producing datasets of unprecedented size and richness, increasing the potential for high-impact scientific discovery. This possibility, coupled with the challenge of exploring a large number of sources, has led to the development of novel machine-learning-based anomaly detection approaches, such as astronomaly. For the first time, we test the scalability of astronomaly by applying it to almost 4 million images of galaxies from the Dark Energy Camera Legacy Survey. We use a trained deep learning algorithm to learn useful representations of the images and pass these to the anomaly detection algorithm isolation forest, coupled with astronomaly’s active learning method, to discover interesting sources.We find that data selection criteria have a significant impact on the trade-off between finding rare sources such as strong lenses and introducing artefacts into the dataset. We demonstrate that active learning is required to identify the most interesting sources and reduce artefacts, while anomaly detection methods alone are insufficient. Using astronomaly, we find 1635 anomalies among the top 2000 sources in the dataset after applying active learning, including 8 strong gravitational lens candidates, 1609 galaxy merger candidates, and 18 previously unidentified sources exhibiting highly unusual morphology. Our results show that by leveraging the human-machine interface, astronomaly is able to rapidly identify sources of scientific interest even in large datasets.

Astronomaly at Scale: Searching for Anomalies Amongst 4 Million Galaxies

Sérgio Sacani

Keller geo edu

hasnakhan12

Genomic Research: The Jump to Light Speed

Larry Smarr

"Some Reflections on Data in the Public Sector" : Communia: The European Them...

Tom Moritz

Google Techtalk 2006

Alberto Conti

An Integrated Science Cyberinfrastructure for Data-Intensive Research

Larry Smarr

The Emerging Cyberinfrastructure for Earth and Ocean Sciences

Larry Smarr

The changing landscape of science

Chelle Gentemann

Big Data in Science

FLARECAST

Data literacy connects the knowledge of deep concepts of statistics, computer science, visualization, and domain science with the practical understanding of the data---and the stories it tells---that we encounter in our lives. As data becomes more pervasive, we see teaching data literacy to students as part of a broad education as an imperative for 21st century. Learning how to arm the next generation with the tools to make the most of data, and avoid the common pitfalls of data, will be a major thrust of our efforts with the Moore/Sloan initiative at Berkeley, through the Berkeley Institute for Data Science.

Computational Training for Domain Scientists & Data Literacy

Joshua Bloom

Toward a Global Interactive Earth Observing Cyberinfrastructure

Larry Smarr

Swells, Soundings, and Sustainability in the Oceans

Dawn Wright

Similaire à Deep Learning in Deep Space (20)

A 100 gigabit highway for science: researchers take a 'test drive' on ani tes...

Efficient data reduction and analysis of DECam images using multicore archite...

Data Science at Berkeley

Joshua Bloom Data Science at Berkeley

Agents In An Exponential World Foster

Surfing the Exponential for Forty Years

Presentation

Keynote on 2015 Yale Day of Data

Astronomaly at Scale: Searching for Anomalies Amongst 4 Million Galaxies

Keller geo edu

Genomic Research: The Jump to Light Speed

"Some Reflections on Data in the Public Sector" : Communia: The European Them...

Google Techtalk 2006

An Integrated Science Cyberinfrastructure for Data-Intensive Research

The Emerging Cyberinfrastructure for Earth and Ocean Sciences

The changing landscape of science

Big Data in Science

Computational Training for Domain Scientists & Data Literacy

Toward a Global Interactive Earth Observing Cyberinfrastructure

Swells, Soundings, and Sustainability in the Oceans

Plus de Universitat Politècnica de Catalunya

This document provides an overview of deep generative learning and summarizes several key generative models including GANs, VAEs, diffusion models, and autoregressive models. It discusses the motivation for generative models and their applications such as image generation, text-to-image synthesis, and enhancing other media like video and speech. Example state-of-the-art models are provided for each application. The document also covers important concepts like the difference between discriminative and generative modeling, sampling techniques, and the training procedures for GANs and VAEs.

Deep Generative Learning for All - The Gen AI Hype (Spring 2024)

Deep Learning in Deep Space

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Deep Learning in Deep Space

Similaire à Deep Learning in Deep Space (20)

Plus de Universitat Politècnica de Catalunya

Plus de Universitat Politècnica de Catalunya (20)

Dernier

Dernier (20)

Deep Learning in Deep Space