Feeding Drupal in Real-Time

•

6 likes•3,226 views

This document discusses using the Feeds module in Drupal to import content into a site in real-time using PubSubHubbub. It explains that Feeds allows importing content from various sources into nodes, users, taxonomy, etc. but typically involves periodic cron jobs. PubSubHubbub allows setting up a publish/subscribe system so that when content is published by a source, like new check-ins on Gowalla, Drupal is notified immediately instead of waiting for a cron job. It demonstrates importing geotagged Flickr photos and seeing real-time Gowalla check-ins on a map as they are published.

Technology

Overview

• What is Feeds module all about?

• How does Feeds module work?

• DEMO: Importing Flickr photos

• Real-time feeds with PubSubHubbub

• Very Exciting Demonstration with Gowalla!

What is Feeds Module?

• Import and aggregation framework

• Creates nodes, users, taxonomy, ﬂat data...

• From RSS / Atom / SOAP / CSV / ...

• Mapping of input to Drupal elements

• It’s easily extensible and supports
PubSubHubbub as a subscriber

But, what about...

Periodic One-oﬀ
Imports Imports

Aggregator Migrate
Feed API Node Import
User Import

How does it work?

Fetcher Parser Processor

How does Feeds work?

Fetcher Parser Processor

HTTP RSS NODE
HTTPS ATOM USER
FILE FLICKR TAXONOMY
... ... ...

How does it work?
Feeds Importer
{
Fetcher

HTTP
HTTPS
FILE
...
Parser

RSS
ATOM
FLICKR
...
Processor

NODE
USER
TAXONOMY
...

Let’s import some stuﬀ
Feeds Importer
{
Fetcher

HTTP
HTTPS
FILE
...
Parser

RSS
ATOM
FLICKR
...
Processor

NODE
USER
TAXONOMY
...

DEMO

• Importing photos from Flickr into Drupal
nodes

• Using location data from GeoRSS feed to
display them on a Google map

• Import set to run on cron every 30 mins.

What’s wrong with that?
Are we there yet?
Are we there yet?

Publisher Subscriber

Yes! Here you are.

Are we there yet?

Problem
Subscriber

Subscriber

Subscriber

Publisher Subscriber

Subscriber

Subscriber

Subscriber

Enter PubSubHubbub

• A publish / subscribe protocol

• Turns Atom and RSS feeds into real-time
streams

Publisher Hub Subscriber

Step 1 of 2: Subscribe

I want to subscribe to
feed X, send updates
to me.

Publisher Hub Subscriber

Are you for real?

Step 2 of 2: Publish
Hey subscribers,
here’s new content for
I have new content feed X
for feed X
Subscriber

Subscriber
Publisher Hub
Subscriber

Subscriber

Let’s import some more
stuﬀ...

• Now in real-time with PuSH

• Check into Gowalla from mobile phone

• See location update on Google map in
Drupal

What just happened?

Publisher Hub Subscriber

Step 1 of 2: Subscribe
• When we created the Gowalla Feed node,
we asked the hub to send us updates.

I want to subscribe to
John’s Gowalla Feed

Super
Publisher
Gowalla Drupal Site
Feedr

Step 2 of 2: Publish

• No polling

• No cron runs

• Real-time update

Super
Publisher
Gowalla Drupal Site
Feedr

References

Feeds Module - http://drupal.org/project/feeds

Flickr Parser - http://drupal.org/project/feeds_ﬂickr

Pubsubhubbub spec - http://pubsubhubbub.googlecode.com/svn/trunk/pubsubhubbub-
core-0.3.html

Superfeedr - http://superfeedr.com/

Federated Social Web Google Group - http://groups.google.com/group/federated-social-
web

Credits

• PubSubHubbub diagrams inspired by Alex
Barth’s presentation at Drupalcon CPH.

• Walled social network image - David
Simonds (The Economist)

Similar to Feeding Drupal in Real-Time

Centralized social networking websites raise scalability issues — due to the growing number of participants — and policy concerns — such as control, privacy and ownership of users’ data. Distributed Social Networks aim to solve those by enabling architectures where people own their data and share it whenever and to whomever they wish. However, the privacy and scalability challenges are still to be tackled. Here, we present a privacy-aware extension to Google’s PubSubHubbub protocol, using Semantic Web technologies, solving both the scalability and the privacy issues in Distributed Social Networks. We enhanced the tradi- tional features of PubSubHubbub in order to allow content publishers to decide whom they want to share their information with, using semantic and dynamic group-based definition. We also present the application of this extension to SMOB (our Semantic Microblogging framework). Yet, our proposal is application agnostic, and can be adopted by any system requiring scalable and privacy-aware content broadcasting.

Privacy Aware Semantic Dissemination

Pavan Kapanipathi

Shortening the feedback loop

Josh Baer

Band of brothers, building scalable social web apps on windows azure with asp...

Marjan Nikolovski

How bigtop leveraged docker for build automation and one click hadoop provis...

Evans Ye

Exploring Google's Social APIs

Ade Oshineye

State of Puppet 2013 - Puppet Camp DC

Puppet

Trend Micro Big Data Platform and Apache Bigtop

Evans Ye

How bigtop leveraged docker for build automation and one click hadoop provis...

Evans Ye

Developing rich multimedia applications with FI-WARE.

Luis Lopez

Habitat Workshop at Velocity London 2017

Mandi Walls

Providing a great media consumption experience to customers is crucial to maximizing audience engagement. To do that, it is important that you make content available for consumption anytime, anywhere, on any device, with a personalized and interactive experience. This session explores the power of big data log analytics (real-time and batched), using technologies like Spark, Shark, Kafka, Amazon Elastic MapReduce, Amazon Redshift and other AWS services. Such analytics are useful for content personalization, recommendations, personalized dynamic ad-insertions, interactivity, and streaming quality. This session also includes a discussion from Netflix, which explores personalized content search and discovery with the power of metadata.

Maximizing Audience Engagement in Media Delivery (MED303) | AWS re:Invent 2013

Amazon Web Services

Zabbix is an opensource monitoring tool that has been rapidly evolving during the last few years. We will talk about the growth of the product and look at it from several perspectives : technical – how Zabbix has developed functionally, important decisions made project management – which processes help to improve the software quality and which ones help less community – how open is Zabbix and how that has changed over the years both towards more and less openness The talk will illustrate points made with examples from the Zabbix community as well as from an extensive Zabbix use at Nokia.

OSMC 2017 | How is Zabbix doing – an outside look by Rihards Olups

NETWAYS

Recipes for Drupal distributions

Lakshmi Narasimhan Parthasarathy

Keynote Puppet Camp San Francisco 2010

Puppet

It's possible to introduce real-time features to PHP applications without deep modifications of the current codebase. Using WAMP you can build distributed systems out of application components which are loosely coupled and communicate in (soft) real-time. There is no need to learn a whole new language, with the implications it has. It also opens the door to write reactive, event-based, distributed architectures and to achieve easier scalability by distributing messages to multiple systems.

Adding Real-time Features to PHP Applications

Ronny López

Collaborative development with git

Joseluis Laso

Version Control With Subversion

Samnang Chhun

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1IPjAIS. Rick Hudson discusses the motivation, performance, and technical challenges of Go's low latency concurrent GC and why the approach fits Go well. Filmed at qconsf.com. Rick Hudson is a member of Google’s Go team. Rick has published papers on language runtimes, memory management, concurrency, synchronization, memory models, and transactional memory.

Go GC: Prioritizing Low Latency and Simplicity

C4Media

As we enter a new age of automation — where every company needs to be able to deliver better software, faster — our goal is to provide the tools you need to iterate faster, ship sooner and deliver more customer value. Earlier this month, we announced brand new products, Puppet Tasks and Puppet Discovery, to give you greater control and end-to-end visibility over your software delivery. We also introduced exciting updates to Puppet Enterprise and a new integration with Splunk. Plus, the continuous delivery automation software company, Distelli, joined the Puppet family! It’s been a big month, and it’s only getting better. Join Ryan Coleman, Director of Product Management, and Alanna Brown, Director of Product Marketing, on 18 October at 10:00 a.m. PT for an in-depth look at what’s new: Puppet Discovery™ is a new offering that lets you see everything you have in real time across your on-premises, cloud and container infrastructure, and know what you need to automate next. Puppet Tasks™, a new family of offerings that encompass both Puppet Bolt™ and Puppet Enterprise Task Management, makes it simple to automate ad hoc tasks, deploy one-off changes, and execute sequenced actions in an imperative way. Our new integration with Splunk provides a unified workflow between the intelligence provided by Splunk and the automation provided by Puppet, giving you the power to turn insights into action faster. With Distelli joining Puppet, we’re uniting the entire software delivery lifecycle, to bring you a platform built for the enterprise, that integrates with a wide variety of tools and helps you avoid vendor lock-in.

Automation for the Modern Enterprise - 18 October 2017

Claire Priester Papas

Picnic Software - Developing a flexible and scalable application

Nick Josevski

Similar to Feeding Drupal in Real-Time (20)

Privacy Aware Semantic Dissemination

Shortening the feedback loop

Band of brothers, building scalable social web apps on windows azure with asp...

How bigtop leveraged docker for build automation and one click hadoop provis...

Exploring Google's Social APIs

State of Puppet 2013 - Puppet Camp DC

Trend Micro Big Data Platform and Apache Bigtop

How bigtop leveraged docker for build automation and one click hadoop provis...

Developing rich multimedia applications with FI-WARE.

Habitat Workshop at Velocity London 2017

Maximizing Audience Engagement in Media Delivery (MED303) | AWS re:Invent 2013

OSMC 2017 | How is Zabbix doing – an outside look by Rihards Olups

Recipes for Drupal distributions

Keynote Puppet Camp San Francisco 2010

Adding Real-time Features to PHP Applications

Collaborative development with git

Version Control With Subversion

Go GC: Prioritizing Low Latency and Simplicity

Automation for the Modern Enterprise - 18 October 2017

Picnic Software - Developing a flexible and scalable application

Recently uploaded

This presentation explores the impact of HTML injection attacks on web applications, detailing how attackers exploit vulnerabilities to inject malicious code into web pages. Learn about the potential consequences of such attacks and discover effective mitigation strategies to protect your web applications from HTML injection vulnerabilities. for more information visit https://bostoninstituteofanalytics.org/category/cyber-security-ethical-hacking/

HTML Injection Attacks: Impact and Mitigation Strategies

Boston Institute of Analytics

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Drew Madelung

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

Imagine a world where information flows as swiftly as thought itself, making decision-making as fluid as the data driving it. Every moment is critical, and the right tools can significantly boost your organization’s performance. The power of real-time data automation through FME can turn this vision into reality. Aimed at professionals eager to leverage real-time data for enhanced decision-making and efficiency, this webinar will cover the essentials of real-time data and its significance. We’ll explore: FME’s role in real-time event processing, from data intake and analysis to transformation and reporting An overview of leveraging streams vs. automations FME’s impact across various industries highlighted by real-life case studies Live demonstrations on setting up FME workflows for real-time data Practical advice on getting started, best practices, and tips for effective implementation Join us to enhance your skills in real-time data automation with FME, and take your operational capabilities to the next level.

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Safe Software

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Edi Saputra

Real Time Object Detection Using Open CV

Khem

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Manulife - Insurer Innovation Award 2024

The Digital Insurer

MINDCTI Revenue Release Quarter One 2024

MIND CTI

Top 10 Most Downloaded Games on Play Store in 2024

SynarionITSolutions

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

Tata AIG General Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

apidays

Recently uploaded (20)