SlideShare a Scribd company logo
1 of 3
Download to read offline
Data Mining with Big Data
ABSTRACT:
Big Data concern large-volume, complex, growing data sets with multiple,
autonomous sources. With the fast development of networking, data storage, and
the data collection capacity, Big Data are now rapidly expanding in all science and
engineering domains, including physical, biological and biomedical sciences. This
paper presents a HACE theorem that characterizes the features of the Big Data
revolution, and proposes a Big Data processing model, from the data mining
perspective. This data-driven model involves demand-driven aggregation of
information sources, mining and analysis, user interest modeling, and security and
privacy considerations. We analyze the challenging issues in the data-driven model
and also in the Big Data revolution.
EXISTING SYSTEM:
➢ The rise of Big Data applications where data collection has grown tremen
dously and is beyond the ability of commonly used software tools to capture,
manage, and process within a “tolerable elapsed time.” The most
fundamental challenge for Big Data applications is to explore the large
volumes of data and extract useful information or knowledge for future
actions. In many situations, the knowledge extraction process has to be very
efficient and close to real time because storing all observed data is nearly
infeasible.
➢ The unprecedented data volumes require an effective data analysis and
prediction platform to achieve fast response and real-time classification for
such Big Data.
DISADVANTAGES OF EXISTING SYSTEM:
 The challenges at Tier I focus on data accessing and arithmetic computing
procedures. Because Big Data are often stored at different locations and data
volumes may continuously grow, an effective computing platform will have
to take distributed large-scale data storage into consideration for computing.
 The challenges at Tier II center around semantics and domain knowledge for
different Big Data applications. Such information can provide additional
benefits to the mining process, as well as add technical barriers to the Big
Data access (Tier I) and mining algorithms (Tier III).
 At Tier III, the data mining challenges concentrate on algorithm designs in
tackling the difficulties raised by the Big Data volumes, distributed data
distributions, and by complex and dynamic data characteristics.
PROPOSED SYSTEM:
➢ We propose a HACE theorem to model Big Data characteristics. The
characteristics of HACH make it an extreme challenge for discovering
useful knowledge from the Big Data.
➢ The HACE theorem suggests that the key characteristics of the Big Data are
1) huge with heterogeneous and diverse data sources, 2) autonomous with
distributed and decentralized control, and 3) complex and evolving in data
and knowledge associations.
➢ To support Big Data mining, high-performance computing platforms are
required, which impose systematic designs to unleash the full power of the
Big Data.
ADVANTAGES OF PROPOSED SYSTEM:
Provide most relevant and most accurate social sensing feedback
to better understand our society at realtime.
SYSTEM ARCHITECTURE:

More Related Content

What's hot

Big Data Challenges, Presented by Wes Caldwell at SolrExchage DC
Big Data Challenges, Presented by Wes Caldwell at SolrExchage DCBig Data Challenges, Presented by Wes Caldwell at SolrExchage DC
Big Data Challenges, Presented by Wes Caldwell at SolrExchage DC
Lucidworks (Archived)
 

What's hot (20)

Data Mining: Applying data mining
Data Mining: Applying data miningData Mining: Applying data mining
Data Mining: Applying data mining
 
Unit 1
Unit 1Unit 1
Unit 1
 
Research Topics in Data Mining
Research Topics in Data MiningResearch Topics in Data Mining
Research Topics in Data Mining
 
data warehousing and data mining
data warehousing and data mining data warehousing and data mining
data warehousing and data mining
 
Image Processing Phd Thesis Projects
Image Processing Phd Thesis ProjectsImage Processing Phd Thesis Projects
Image Processing Phd Thesis Projects
 
Data Mining with big data total ieee project and entire files.
Data Mining with big data total ieee project and entire files.Data Mining with big data total ieee project and entire files.
Data Mining with big data total ieee project and entire files.
 
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
 
Data mining introduction
Data mining introductionData mining introduction
Data mining introduction
 
Data mining
Data miningData mining
Data mining
 
Big Data Challenges, Presented by Wes Caldwell at SolrExchage DC
Big Data Challenges, Presented by Wes Caldwell at SolrExchage DCBig Data Challenges, Presented by Wes Caldwell at SolrExchage DC
Big Data Challenges, Presented by Wes Caldwell at SolrExchage DC
 
Data mining
Data miningData mining
Data mining
 
Big Data Europe SC6 WS 3: Ron Dekker, Director CESSDA European Open Science A...
Big Data Europe SC6 WS 3: Ron Dekker, Director CESSDA European Open Science A...Big Data Europe SC6 WS 3: Ron Dekker, Director CESSDA European Open Science A...
Big Data Europe SC6 WS 3: Ron Dekker, Director CESSDA European Open Science A...
 
Oracle NoSQL DB & InfiniteGraph - Trends in Big Data and Graph Technology
Oracle NoSQL DB & InfiniteGraph - Trends in Big Data and Graph TechnologyOracle NoSQL DB & InfiniteGraph - Trends in Big Data and Graph Technology
Oracle NoSQL DB & InfiniteGraph - Trends in Big Data and Graph Technology
 
Data Mining: Classification and analysis
Data Mining: Classification and analysisData Mining: Classification and analysis
Data Mining: Classification and analysis
 
9 Data Mining Challenges From Data Scientists Like You
9 Data Mining Challenges From Data Scientists Like You9 Data Mining Challenges From Data Scientists Like You
9 Data Mining Challenges From Data Scientists Like You
 
Big Data Europe SC6 WS 3: Where we are and are going for Big Data in OpenScie...
Big Data Europe SC6 WS 3: Where we are and are going for Big Data in OpenScie...Big Data Europe SC6 WS 3: Where we are and are going for Big Data in OpenScie...
Big Data Europe SC6 WS 3: Where we are and are going for Big Data in OpenScie...
 
Tutorial on Text Mining, ECML, 2002
Tutorial on Text Mining, ECML, 2002Tutorial on Text Mining, ECML, 2002
Tutorial on Text Mining, ECML, 2002
 
Web Mining Project Ideas
Web Mining Project IdeasWeb Mining Project Ideas
Web Mining Project Ideas
 
1 introduction
1 introduction1 introduction
1 introduction
 
Computing with large datasets
Computing with large datasetsComputing with large datasets
Computing with large datasets
 

Similar to Datamining with big data

Similar to Datamining with big data (20)

JPJ1417 Data Mining With Big Data
JPJ1417   Data Mining With Big DataJPJ1417   Data Mining With Big Data
JPJ1417 Data Mining With Big Data
 
A Model Design of Big Data Processing using HACE Theorem
A Model Design of Big Data Processing using HACE TheoremA Model Design of Big Data Processing using HACE Theorem
A Model Design of Big Data Processing using HACE Theorem
 
Big data storage
Big data storageBig data storage
Big data storage
 
A Survey on Big Data Mining Challenges
A Survey on Big Data Mining ChallengesA Survey on Big Data Mining Challenges
A Survey on Big Data Mining Challenges
 
A Review Paper on Big Data and Hadoop for Data Science
A Review Paper on Big Data and Hadoop for Data ScienceA Review Paper on Big Data and Hadoop for Data Science
A Review Paper on Big Data and Hadoop for Data Science
 
Real World Application of Big Data In Data Mining Tools
Real World Application of Big Data In Data Mining ToolsReal World Application of Big Data In Data Mining Tools
Real World Application of Big Data In Data Mining Tools
 
Big data Mining Using Very-Large-Scale Data Processing Platforms
Big data Mining Using Very-Large-Scale Data Processing PlatformsBig data Mining Using Very-Large-Scale Data Processing Platforms
Big data Mining Using Very-Large-Scale Data Processing Platforms
 
Seminar presentation
Seminar presentationSeminar presentation
Seminar presentation
 
A Novel Framework for Big Data Processing in a Data-driven Society
A Novel Framework for Big Data Processing in a Data-driven SocietyA Novel Framework for Big Data Processing in a Data-driven Society
A Novel Framework for Big Data Processing in a Data-driven Society
 
pole2016-A-Recent-Study-of-Emerging-Tools.pdf
pole2016-A-Recent-Study-of-Emerging-Tools.pdfpole2016-A-Recent-Study-of-Emerging-Tools.pdf
pole2016-A-Recent-Study-of-Emerging-Tools.pdf
 
Data minig with Big data analysis
Data minig with Big data analysisData minig with Big data analysis
Data minig with Big data analysis
 
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
 
Conclusion
ConclusionConclusion
Conclusion
 
6.a survey on big data challenges in the context of predictive
6.a survey on big data challenges in the context of predictive6.a survey on big data challenges in the context of predictive
6.a survey on big data challenges in the context of predictive
 
research publish journal
research publish journalresearch publish journal
research publish journal
 
research publish journal
research publish journalresearch publish journal
research publish journal
 
20211011112936_PPT01-Introduction to Big Data.pptx
20211011112936_PPT01-Introduction to Big Data.pptx20211011112936_PPT01-Introduction to Big Data.pptx
20211011112936_PPT01-Introduction to Big Data.pptx
 
ISSUES, CHALLENGES, AND SOLUTIONS: BIG DATA MINING
ISSUES, CHALLENGES, AND SOLUTIONS: BIG DATA MININGISSUES, CHALLENGES, AND SOLUTIONS: BIG DATA MINING
ISSUES, CHALLENGES, AND SOLUTIONS: BIG DATA MINING
 
Issues, challenges, and solutions
Issues, challenges, and solutionsIssues, challenges, and solutions
Issues, challenges, and solutions
 
FR.pptx
FR.pptxFR.pptx
FR.pptx
 

More from muhammed jassim k

More from muhammed jassim k (20)

Adapter Wavelet Thresholding for Image Denoising Using Various Shrinkage Unde...
Adapter Wavelet Thresholding for Image Denoising Using Various Shrinkage Unde...Adapter Wavelet Thresholding for Image Denoising Using Various Shrinkage Unde...
Adapter Wavelet Thresholding for Image Denoising Using Various Shrinkage Unde...
 
Image Cryptography using Nearest Prime Pixels
Image Cryptography using Nearest Prime PixelsImage Cryptography using Nearest Prime Pixels
Image Cryptography using Nearest Prime Pixels
 
Cloud armor:Supporting Reputation-Based Trust Management for Cloud Service
Cloud armor:Supporting Reputation-Based Trust Management for Cloud ServiceCloud armor:Supporting Reputation-Based Trust Management for Cloud Service
Cloud armor:Supporting Reputation-Based Trust Management for Cloud Service
 
ELECTRONIC PROTECTION FOR EXAM PAPER LEAKAGE
ELECTRONIC PROTECTION FOR EXAM PAPER LEAKAGEELECTRONIC PROTECTION FOR EXAM PAPER LEAKAGE
ELECTRONIC PROTECTION FOR EXAM PAPER LEAKAGE
 
4.weather based smart watering system using soil sensor and gsm
4.weather based smart watering system using soil sensor and gsm4.weather based smart watering system using soil sensor and gsm
4.weather based smart watering system using soil sensor and gsm
 
26. qo s ranking prediction for cloud services
26. qo s ranking prediction for cloud services26. qo s ranking prediction for cloud services
26. qo s ranking prediction for cloud services
 
Energy-Efficient intelligent street lighting system using traffic-adaptive co...
Energy-Efficient intelligent street lighting system using traffic-adaptive co...Energy-Efficient intelligent street lighting system using traffic-adaptive co...
Energy-Efficient intelligent street lighting system using traffic-adaptive co...
 
Fire col a collaborative protection
Fire col a collaborative protectionFire col a collaborative protection
Fire col a collaborative protection
 
privacy preserving abstract
 privacy preserving abstract privacy preserving abstract
privacy preserving abstract
 
33. dynamic resource allocation using virtual machines
33. dynamic resource allocation using virtual machines33. dynamic resource allocation using virtual machines
33. dynamic resource allocation using virtual machines
 
An automated dynamic offset for network selection in heterogeneous networks
An automated dynamic offset for network selection in heterogeneous networksAn automated dynamic offset for network selection in heterogeneous networks
An automated dynamic offset for network selection in heterogeneous networks
 
ALTERDROID:Differential fault Analysis of Obfuscated Smartphone Malware
ALTERDROID:Differential fault Analysis of Obfuscated Smartphone MalwareALTERDROID:Differential fault Analysis of Obfuscated Smartphone Malware
ALTERDROID:Differential fault Analysis of Obfuscated Smartphone Malware
 
A location-and Diversity aware News feed system for mobile user
A location-and Diversity aware News feed system for mobile userA location-and Diversity aware News feed system for mobile user
A location-and Diversity aware News feed system for mobile user
 
A feature-Enriched Completely Blind image Quality Evaluator
A feature-Enriched Completely Blind image Quality EvaluatorA feature-Enriched Completely Blind image Quality Evaluator
A feature-Enriched Completely Blind image Quality Evaluator
 
A cloud
A cloudA cloud
A cloud
 
Hierarchical supervisory control system for pe vs participating in frequency ...
Hierarchical supervisory control system for pe vs participating in frequency ...Hierarchical supervisory control system for pe vs participating in frequency ...
Hierarchical supervisory control system for pe vs participating in frequency ...
 
On demand retrieval of crowdsourced
On demand retrieval of crowdsourcedOn demand retrieval of crowdsourced
On demand retrieval of crowdsourced
 
Medical warehouse business distribution
Medical warehouse business distributionMedical warehouse business distribution
Medical warehouse business distribution
 
Discoveringlatentsemanticsinweb 160617093617
Discoveringlatentsemanticsinweb 160617093617Discoveringlatentsemanticsinweb 160617093617
Discoveringlatentsemanticsinweb 160617093617
 
Raspberrypiprojectsforeceeee 150724094838-lva1-app6891
Raspberrypiprojectsforeceeee 150724094838-lva1-app6891Raspberrypiprojectsforeceeee 150724094838-lva1-app6891
Raspberrypiprojectsforeceeee 150724094838-lva1-app6891
 

Recently uploaded

Gardella_PRCampaignConclusion Pitch Letter
Gardella_PRCampaignConclusion Pitch LetterGardella_PRCampaignConclusion Pitch Letter
Gardella_PRCampaignConclusion Pitch Letter
MateoGardella
 
An Overview of Mutual Funds Bcom Project.pdf
An Overview of Mutual Funds Bcom Project.pdfAn Overview of Mutual Funds Bcom Project.pdf
An Overview of Mutual Funds Bcom Project.pdf
SanaAli374401
 
Gardella_Mateo_IntellectualProperty.pdf.
Gardella_Mateo_IntellectualProperty.pdf.Gardella_Mateo_IntellectualProperty.pdf.
Gardella_Mateo_IntellectualProperty.pdf.
MateoGardella
 
Making and Justifying Mathematical Decisions.pdf
Making and Justifying Mathematical Decisions.pdfMaking and Justifying Mathematical Decisions.pdf
Making and Justifying Mathematical Decisions.pdf
Chris Hunter
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global Impact
PECB
 

Recently uploaded (20)

Holdier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfHoldier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdf
 
Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104
 
Paris 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityParis 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activity
 
Gardella_PRCampaignConclusion Pitch Letter
Gardella_PRCampaignConclusion Pitch LetterGardella_PRCampaignConclusion Pitch Letter
Gardella_PRCampaignConclusion Pitch Letter
 
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptxBasic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
 
An Overview of Mutual Funds Bcom Project.pdf
An Overview of Mutual Funds Bcom Project.pdfAn Overview of Mutual Funds Bcom Project.pdf
An Overview of Mutual Funds Bcom Project.pdf
 
This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.
 
PROCESS RECORDING FORMAT.docx
PROCESS      RECORDING        FORMAT.docxPROCESS      RECORDING        FORMAT.docx
PROCESS RECORDING FORMAT.docx
 
Advance Mobile Application Development class 07
Advance Mobile Application Development class 07Advance Mobile Application Development class 07
Advance Mobile Application Development class 07
 
Gardella_Mateo_IntellectualProperty.pdf.
Gardella_Mateo_IntellectualProperty.pdf.Gardella_Mateo_IntellectualProperty.pdf.
Gardella_Mateo_IntellectualProperty.pdf.
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
 
Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1
 
Making and Justifying Mathematical Decisions.pdf
Making and Justifying Mathematical Decisions.pdfMaking and Justifying Mathematical Decisions.pdf
Making and Justifying Mathematical Decisions.pdf
 
SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...
SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...
SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...
 
APM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across SectorsAPM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across Sectors
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global Impact
 
Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...
Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...
Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...
 
Grant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingGrant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy Consulting
 
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
 
Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..
 

Datamining with big data

  • 1. Data Mining with Big Data ABSTRACT: Big Data concern large-volume, complex, growing data sets with multiple, autonomous sources. With the fast development of networking, data storage, and the data collection capacity, Big Data are now rapidly expanding in all science and engineering domains, including physical, biological and biomedical sciences. This paper presents a HACE theorem that characterizes the features of the Big Data revolution, and proposes a Big Data processing model, from the data mining perspective. This data-driven model involves demand-driven aggregation of information sources, mining and analysis, user interest modeling, and security and privacy considerations. We analyze the challenging issues in the data-driven model and also in the Big Data revolution. EXISTING SYSTEM: ➢ The rise of Big Data applications where data collection has grown tremen dously and is beyond the ability of commonly used software tools to capture, manage, and process within a “tolerable elapsed time.” The most fundamental challenge for Big Data applications is to explore the large volumes of data and extract useful information or knowledge for future actions. In many situations, the knowledge extraction process has to be very efficient and close to real time because storing all observed data is nearly infeasible. ➢ The unprecedented data volumes require an effective data analysis and prediction platform to achieve fast response and real-time classification for such Big Data.
  • 2. DISADVANTAGES OF EXISTING SYSTEM:  The challenges at Tier I focus on data accessing and arithmetic computing procedures. Because Big Data are often stored at different locations and data volumes may continuously grow, an effective computing platform will have to take distributed large-scale data storage into consideration for computing.  The challenges at Tier II center around semantics and domain knowledge for different Big Data applications. Such information can provide additional benefits to the mining process, as well as add technical barriers to the Big Data access (Tier I) and mining algorithms (Tier III).  At Tier III, the data mining challenges concentrate on algorithm designs in tackling the difficulties raised by the Big Data volumes, distributed data distributions, and by complex and dynamic data characteristics. PROPOSED SYSTEM: ➢ We propose a HACE theorem to model Big Data characteristics. The characteristics of HACH make it an extreme challenge for discovering useful knowledge from the Big Data. ➢ The HACE theorem suggests that the key characteristics of the Big Data are 1) huge with heterogeneous and diverse data sources, 2) autonomous with distributed and decentralized control, and 3) complex and evolving in data and knowledge associations.
  • 3. ➢ To support Big Data mining, high-performance computing platforms are required, which impose systematic designs to unleash the full power of the Big Data. ADVANTAGES OF PROPOSED SYSTEM: Provide most relevant and most accurate social sensing feedback to better understand our society at realtime. SYSTEM ARCHITECTURE: