MongoDB World 2018: Tutorial - Got Dibs? Building a Real-Time Bidding App with Change Streams

While you wait…
...let’s get some things out of the way
• Install MongoDB 3.6 or 4.0
• https://docs.mongodb.com/manual/installation/#mongodb-community-edition
• Install Node 8 or Node 10
• https://nodejs.org/en/
• Download the code we’ll start with, and unzip it
• https://github.com/hswolff/dibs/archive/tutorial-start.zip
• Install all dependencies
• cd client && npm install && cd ../server && npm install

Got Dibs? Building a
Real-Time Bidding App
with Change Streams
A tutorial by Harry Wolff

Harry Wolff
Lead Engineer, MongoDB
@hswolff

Building a Real-Time Bidding App
with Change Streams
BUT FIRST

Build an app
Easy and fun to use
Has to be real time
And we need it now!

Application Architecture
API
Server
(Node)
MongoDB
Client
(React)

Outline
1. Prerequisites
2. Create the Server
3. Create the Client
4. Let's call dibs
5. To real time, and beyond!

Outline
1. Prerequisites
a. Set up MongoDB
b. Create server folder
c. Connect to MongoDB
d. Set up our API
e. Create API endpoints
a. Create client folder
b. Connect to our API
c. Render all dibs
d. Add support for creating a dib
e. Enable a user to log in
4. Let's call dibs
a. Server support
b. Client support
a. Server support
b. Client support

Outline
1. Prerequisites
a. Set up MongoDB
b. Create server folder
c. Connect to MongoDB
d. Set up our API
e. Create API endpoints
a. Create client folder
b. Connect to our API
c. Render all dibs
d. Add support for creating a dib
e. Enable a user to log in
4. Let's call dibs
a. Server support
b. Client support
a. Server support
b. Client support
Change
Streams

Source Code
We’re starting with a partially complete
codebase.
You’ll see TODO comments throughout.
By the end of today all of those will be
completed.

Source Code
All source code for each step is
available online for quick reference.
https://github.com/hswolff/dibs/tree/tutorial
Download a copy of the code for easy reference
https://github.com/hswolff/dibs/archive/tutorial.zip

Download a copy of these slides
http://bit.ly/mdbw18-dibs
Slides

# Create the folder where the database will save its data.
mkdir db
# Start the mongo daemon, and leave it open in a terminal.
mongod --port 27017 --dbpath $(pwd)/db --replSet rs0
# Instantiate the replica set. Only need to run this one time.
mongo --eval "rs.initiate()"
Set up MongoDB

Set up MongoDB
# Connect to the db from a different shell.
mongo
# You should see
rs0:PRIMARY>

open server/db.js
Connect to MongoDB

function LetsCode(){}
Connect to MongoDB

1. Open node REPL
a. node
2. Import db file and connect to MongoDB
a. const db = require('./db);
b. db.connect();
3. Find all documents
a. db.Models.Dib.find().then(console.log);
4. Create a document
a. db.Models.Dib.create({ title: 'First Item', creator: 'Yay'
}).then(console.log);
5. Find all documents again
Connect to MongoDB: Let’s test it

1. Try and create a document that doesn’t
pass validation.
2. Use the Mongoose method to find one
document
(hint http://mongoosejs.com/docs/queries.html)
Connect to MongoDB: Let’s test it

https://github.com/hswolff/dibs/blob/tutori
al/step2a-d/server/db.js
Connect to MongoDB: Catch Up

Set up our API
open server/server.js

Set up our API

const express = require('express');
const cors = require('cors');
const bodyParser = require('body-parser');
const PORT = process.env.PORT || 8080;
const DOMAIN = process.env.DOMAIN || 'http://localhost' ;
Set up our API: server.js

1. Start server
a. node server.js
2. Open browser: http://localhost:8080/
3. What do you see?
Set up our API: Test It

1. Change a handler to return JSON to the
browser
2. Add a new handler that says ‘Goodbye’
3. Add a handler that takes in a parameter
so we can say `Hello ${name}`
(hint: https://expressjs.com/en/guide/routing.html)
Set up our API: Exercise

https://github.com/hswolff/dibs/blob/tutori
al/step2a-d/server/server.js
Set up our API: Catch up

Create API endpoints
# We want to expose our MongoDB data
through an API
# We’re going to expose all API methods
under an /api route namespace.
open server/api.js

Create API endpoints

// Create a Dib
api.post('/dibs', async (req, res) => {
const { creator, title } = req.body;
if (!creator || !title) {
return res.status(500).json({
error: 'Must include a "creator" and a
"title".',
});
}
res.json({
data: await Models.Dib.create({
creator ,
title,
}),
});
});
return api;
};
Create API endpoints: api.js

Check GET and browse to:
http://localhost:8080/api/dibs
Check POST with cURL
curl --data '{"title": "Hello from Curl", "creator": "Mr. Curly"}'
-X POST --header "Content-Type: application/json"
http://localhost:8080/api/dibs
Create API endpoints: Test It

Create API endpoints: Catch Up
https://github.com/hswolff/dibs/tree/tutori
al/step2e/server

Create client folder
We used Create React App to scaffold our
client

Create client folder: Test It
cd client
npm start
Does your browser open?

Connect to our API
open src/services/api.js

Connect to our API

Connect to our API: src/services/api.js
const baseUrl = 'http://localhost:8080' ;
function fetchWrapper(apiPath, options) {
return fetch(`${baseUrl}/api${apiPath}`, options).then(res => res.json());
}

export default {
getDibs() {
return fetchWrapper('/dibs');
},
createDib({ creator, title }) {
return fetchWrapper('/dibs', {
method: 'POST',
headers: {
'content-type': 'application/json',
},
body: JSON.stringify({
creator,
title,
}),
});
},
};
Connect to our API: src/services/api.js

1. Ensure your API server is running.
2. Ensure your client environment is running.
3. Go to client/src/index.js.
4. We’re going to expose our client API globally.
Connect to our API: Test It

import api from './services/api';
window.api = api;
Connect to our API: client/src/index.js

5. Go to your browser and open the Developer Tools
(for Chrome go to View > Developer > Developer Tools)
6. Go to the “Console” tab
7. Type ‘api’ in the console. You should see your API code!
8. Exercise: Try making a request for “getDibs”
Answer: api.getDibs().then(console.log)
9. Exercise: Try creating a new dib
Answer: api.createDib({ creator: 'Browser', title: 'From
the client!' }).then(console.log)
Connect to our API: Test It

Remove our debug code from index.js
Connect to our API: Cleanup

https://github.com/hswolff/dibs/blob/tutorial/ste
p3a-c/client/src/services/api.js
Connect to our API: Catch up

# We want to fetch our data from our server API
# We’re going to load the data in our HomePage
# We’re going to use React’s lifecycle methods to do so
# Namely the componentDidMount lifecycle method
open src/components/HomePage.js
Render all dibs: Load the data

Render all dibs

import React, { Component } from 'react';
export default class HomePage extends Component {
render() {
return (
<div>
<header>
<h1>Got Dibs?</h1>
</header>
</div>
);
}
}
Render all dibs: components/HomePage.js

import './App.css';
import HomePage from './components/HomePage';
class App extends Component {
render() {
return (
<div className="App-content">
<HomePage />
</div>
);
}
}
export default App;
Render all dibs: src/App.js

.App-content {
width: 600px;
margin: 20px auto;
}
Render all dibs: src/App.css

import api from '../services/api';
export default class HomePage extends Component {
state = {
dibs: [],
};
componentDidMount = async () => {
const result = await api.getDibs();
this.setState({ dibs: result.data });
};
render() {
const { dibs } = this.state;
return (
<div>
<header>
<h1>Got Dibs?</h1>
</header>
<div>
{dibs.map(dib => (
<pre key={dib._id}>
{JSON.stringify(dib, null, 2)}
</pre>
))}
</div>
</div>
);
}

# Now we want to render each item a little better
# To do so we’ll use the component “DibCell”
open components/DibCell.js
Render all dibs: With style!

Render all dibs: With style!

import './DibCell.css';
export default class DibCell extends Component {
render() {
const { title, creator } = this.props;
return (
<div className="dib-container">
<div className="dib-left">
<div className="dib-creator">
<b>{creator}</b> is offering
</div>
<div className="dib-title">{title}</div>
</div>
<div className="dib-right" />
</div>
);
}
}
Render all dibs: components/DibCell.js

.dib-container {
margin: 20px 0;
padding: 20px 0;
display: flex;
}
.dib-left {
width: 70%;
padding-right: 20px;
margin-right: 20px;
display: flex;
flex-direction: column;
justify-content: space-between;
}
.dib-creator {
font-size: 0.9rem;
}
.dib-title {
font-size: 2rem;
}
.dib-right {
width: 30%;
display: flex;
flex-direction: column;
justify-content: space-between;
}
Render all dibs: components/DibCell.css

import DibCell from './DibCell';
render() {
return (
<div>
<header>
<h1>Got Dibs?</h1>
</header>
<div>{dibs.map(dib => <DibCell key={dib._id} {...dib} />)}</div>
</div>
);
}

https://github.com/hswolff/dibs/tree/tuto
rial/step3a-c/client/src/components
Render all dibs: Catch up

Add support for creating
a dib

# Using our CreateNewDib component
# We need to hook it up to actually create the new dib.
open src/components/CreateNewDib.js
Add support for creating a dib

Add support for creating a dib

export default class CreateNewDib extends Component {
state = {
title: '',
error: null,
};
render() {
const { error, title } = this.state;
return (
<div>
{error}
<textarea
autoFocus
tabIndex={1}
placeholder="Create New Dib"
value={title}
onChange={e => this.setState({ title:
e.target.value })}
/>
<br />
<button>Create New Dib</button>
</div>
);
}
}
Add support for creating a dib: CreateNewDib

import CreateNewDib from './CreateNewDib';
render() {
return (
<div>
<header>
<h1>Got Dibs?</h1>
</header>
<CreateNewDib />
<div>{dibs.map(dib => <DibCell key={dib._id} {...dib} />)}</div>
</div>
);
}
Add support for creating a dib: HomePage

import api from '../services/api';
<button onClick={this.createNew}>Create New Dib</button>
createNew = async e => {
e.preventDefault();
this.setState({ error: null });
try {
await api.createDib({
title: this.state.title,
creator: 'Fake Creator',
});
this.setState({ title: '' });
this.props.onSuccess();
} catch (error) {
this.setState({ error: error.message });
}
};
Add support for creating a dib: CreateNewDib

componentDidMount = () => {
this.loadDibs();
};
loadDibs = async () => {
const result = await api.getDibs();
this.setState({ dibs: result.data });
};
<CreateNewDib onSuccess={this.loadDibs} />
Add support for creating a dib: HomePage

1. Try creating a new dib!
2. Does your page refresh with the new dib?
Add support for creating a dib: Test It

1. Style CreateNewDib UI
2. Add a button on the HomePage which shows and hides the
CreateNewDib component
Add support for creating a dib: Exercise

rial/step3d/client/src/components
Add support for creating a dib: Catch Up

Enable a user to log in
# We need to enable a user to give us a username so we can
show the right person that created a dib
# We store the user in localStorage to persist it between
refreshes
# We’re going to use the SignIn component to let a user sign
in.
open src/components/SignIn.js

Enable a user to log in

export default class SignIn extends Component {
state = {
input: '',
};
logIn = () => {
this.props.onUsernameChange(this.state.input);
};
logOut = () => {
this.props.onUsernameChange(null);
};
render() {
const { username } = this.props;
const { input } = this.state;
if (username) {
return (
<div>
Logged in as: < b>{username}</b>
<button onClick={this.logOut}>Log Out</ button>
</div>
);
}
return (
<div>
<input
value={input}
onChange={e => this.setState({ input: e.target.value })}
/>
<button onClick={this.logIn}>Log In</button>
</div>
);
}
}
Enable a user to log in: SignIn.js

import SignIn from './SignIn';
state = {
dibs: [],
username: localStorage.getItem('username'),
};
onUsernameChange = username => {
localStorage.setItem('username', username);
this.setState({ username });
};
render() {
const { dibs, username } = this.state;
return (
<div>
<header>
<h1>Got Dibs?</h1>
</header>
<SignIn username={username}
onUsernameChange={this.onUsernameChange} />
<br />
<CreateNewDib onSuccess={this.loadDibs}
username={username} />
<div>{dibs.map(dib => <DibCell key={dib._id} {...dib}
/>)}</div>
</div>
);
Enable a user to log in: HomePage.js

createNew = async e => {
e.preventDefault();
try {
await api.createDib({
title: this.state.title,
creator: this.props.username,
});
this.setState({ title: '' });
this.props.onSuccess();
} catch (error) {
this.setState({ error: error.message });
}
};
Enable a user to log in: CreateNewDib.js

1. Try creating a new dib!
2. Do you see it appear?
Enable a user to log in: Test it

1. Only show CreateNewDib if a username is set
2. Show an error if a user puts in an empty username
3. Style SignIn component
Enable a user to log in: Exercise

rial/step3e/client/src/components
Enable a user to log in: Catch Up

# We need to extend our Schema to support claiming a Dib
Server support

Server support: Schema

Server support: server/db.js
const dibScheme = new mongoose.Schema(
{
title: {
type: String,
required: true,
},
creator: {
type: String,
required: true,
},
claimed: { // New
user: String,
time: Date,
},
},
{ timestamps: true }
);
const Dib = mongoose.model('Dib', dibScheme);

Server support
# We need to add a new endpoint
# Endpoint adds support for a user to claim a dib
# We need to validate the data sent
Need to make sure all data required is sent
Need to make sure that the dib exists
Need to make sure a dib has not already been claimed

Server support: Endpoint

// Claim a Dib
api.put('/dibs/:dibId', async (req, res) => {
const { dibId } = req.params;
const { user } = req.body;
if (!user) {
error: 'No "user" given to claim Dib.',
});
}
let dib;
try {
dib = await Models.Dib.findById(dibId);
} catch (error) {
error: 'Cannot find Dib with given id',
});
}
Server support: server/api.js
if (dib.claimed.user) {
error: 'Dib has already been claimed!',
});
}
dib.claimed.user = user;
dib.claimed.time = new Date();
res.json({
data: await dib.save(),
});
});

Server support: Catch Up
rial/step4/server

# Add a new method to our API service to call our new API
endpoint called “claimDib”
Client support

Client support

Client support: client/src/services/api.js
claimDib({ id, user }) {
return fetchWrapper(`/dibs/${id}`, {
method: 'PUT',
headers: {
'content-type' : 'application/json' ,
},
body: JSON.stringify({
user,
}),
});
},

# Add a handler to actually make our API call to claim a dib
# Update our “Dibs?” text to say “Claimed!” if the dib is
already taken
open src/components/DibCell.js
Client support

.dib-claim-button {
height: 100%;
width: 100%;
font-size: 50px;
cursor: pointer;
}
Client support: client/src/DibCell.js
const { title, creator, claimed } = this.props;
const isClaimed = claimed && claimed.user != null;
return (
<div className="dib-container">
<div className="dib-left">
<div className="dib-creator">
<b>{creator}</b> is offering
</div>
<div className="dib-title">{title}</div>
</div>
<div className="dib-right">
<button className="dib-claim-button">
{isClaimed ? 'Claimed!' : 'Dibs?'}
</button>
</div>
</div>
);

Client support: client/src/DibCell.js
<button className="dib-claim-button" onClick={this.claimDib}>
{isClaimed ? 'Claimed!' : 'Dibs?'}
</button>
claimDib = async () => {
try {
await api.claimDib({
id: this.props._id,
user: this.props.username,
});
} catch (error) {
console.error('Error claiming dib', error);
}
};

Client support: client/src/HomePage.js
<div>
{dibs.map(dib => (
<DibCell key={dib._id} {...dib} username={username} />
))}
</div>

1. Try claiming a dib!
2. Refresh the page
3. Does it show that it’s claimed?
Client support: Try It

1. Show an error if a dib has already been claimed
2. Disable the button if a user is not logged in
Client support: Exercise

rial/step4/client/src
Client support: Catch Up

# We want to open a Change Stream to be notified of when our
collection in MongoDB changes
# Change Streams are a new feature in MongoDB 3.6
# Change Streams allow applications to access real-time data
changes without the complexity and risk of tailing the oplog.
(docs)
Server support

Server support

Server support: db.js
async function connect() {
const dibChangeStream = Models.Dib.collection.watch({
fullDocument: 'updateLookup',
});
dibChangeStream.on('change', result => {
console.log(JSON.stringify(result, null, 2));
});
}

# Try making a new dib!
# Try claiming a dib!
# Should see output from your server.
Server support: Test Change Stream

So let’s talk real time.
We want our updates to show immediately.
There’s one technology good for that…
WebSockets!

WebSockets are an advanced technology that makes it possible to
open an interactive communication session between the user's
browser and a server. With this API, you can send messages to a
server and receive event-driven responses without having to poll
the server for a reply.
- https://developer.mozilla.org/en-US/docs/Web/API/WebSockets
_API

# We’re using socket.io as our WebSocket server
# Create our WebSocket server
Server support

Server support: server.js
const http = require('http');
const socketIo = require('socket.io');
// ...
const httpServer = http.Server(app);
const io = socketIo(httpServer);
// ...
await db.connect(io);
// ...
httpServer.listen(PORT, () => {
console.log(`Server ready at ${DOMAIN}:${PORT}/`);
});

# Now we want to share those changes to the client
# And we’ll do that through our WebSocket
Server support

Server support: db.js
async function connect(io) {
const dibChangeStream = Models.Dib.collection.watch({
fullDocument: 'updateLookup',
});
dibChangeStream.on('change', result => {
io.emit('dib changeEvent', {
type: result.operationType,
dib: {
claimed: {},
...result.fullDocument,
id: result.fullDocument._id,
},
});
});
}

rial/step5/server
Server support: Catch Up

# We need to be able to connect to our WebSocket server
# To do that we’ll use…
# socket.io on the client!
Client support

# Let’s add a method for us to connect to our WebSocket server
# In particular we want to subscribe to our ‘dib changeEvent’
# When that event happens we want our callback to be called
open src/services/api.js
Client support

Client support: client/src/services/api.js
import io from 'socket.io-client';
function createSocket() {
const socket = io(baseUrl);
return socket;
}
subscribeToDibChanges(cb) {
createSocket().on('dib changeEvent', cb);
},

# Now let’s actually subscribe to the WebSocket
# We’ll update our UI when a new event comes in
Client support

componentDidMount = () => {
this.loadDibs();
api.subscribeToDibChanges (event => {
const { dib, type } = event;
const currentDibs = this.state.dibs;
if (type === 'update') {
return this.setState({
dibs: [
... currentDibs .map(prevDib => {
if (prevDib._id === dib._id) {
return dib;
}
return prevDib;
}),
],
});
}
// If adding a new dib
return this.setState({
dibs: [dib, ...currentDibs],
});
});
};
Client support: components/HomePage.js

1. Open another browser in Incognito
2. Try making a new dib.
3. Do you see it show up in the other browser??
Client support: Test it!

rial/step5/client/src
Client support: Catch Up

1. Instantiated our own local MongoDB
2. Created an entire API server, powered by Express
3. Opened a MongoDB Change Stream
4. Created a WebSocket server that tells the client when data
in our MongoDB Change Stream changed
5. Created a React application
6. Connected to our API server on the client
7. Added the ability to list all our dibs
8. Added the ability to create a dib
9. Added the ability to claim a dib
10. ...and we made it all work in real time!
Let’s Recap

1. Location of Dib.
2. Price/bounty of Dib.
3. Auth with Google
4. Real-time comments on a Dib
5. Able to edit Dibs that you created.
6. Able to un-claim a Dib if you're the creator of the dib.
7. ...and more!
Take it further

Harry Wolff
Lead Engineer, MongoDB
@hswolff
TheConsoleLog.com

MongoDB World 2018: Tutorial - Got Dibs? Building a Real-Time Bidding App with Change Streams

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à MongoDB World 2018: Tutorial - Got Dibs? Building a Real-Time Bidding App with Change Streams

Similaire à MongoDB World 2018: Tutorial - Got Dibs? Building a Real-Time Bidding App with Change Streams (20)

Plus de MongoDB

Plus de MongoDB (20)

Dernier

Dernier (20)

MongoDB World 2018: Tutorial - Got Dibs? Building a Real-Time Bidding App with Change Streams