The document describes building a link shortening application called YA.LS using Amazon Web Services. It discusses:
1) Developing the application using Node.js, Express, and other packages and deploying it to AWS Elastic Beanstalk.
2) Storing shortened links and target URLs in DynamoDB and caching them locally.
3) Redirecting requests using the stored URLs.
4) Enhancing it to take screenshots, store them in S3, and record analytics using SQS, Lambda, and Kinesis.
5) Configuring authentication with IAM and deploying the infrastructure with CloudFormation.
2. AudienceProject
• Developing products that allow to learn digital audiences
• Started using AWS more than 5 years ago
• Fully migrated to AWS more than 2 years ago
• Processing 4 billions requests monthly
• Generating reports based on 8 billions of requests with batched reports
• Online reports on 300 millions of records
• Used ~40% of services provided by AWS
• Totally happy regarding using AWS
4. Amazon provides Infrastructure as a Services
• Virtual server (EC2) with operating system preinstalled
- Wide choice of instance types optimized for CPU, RAM or I/O
- Preinstalled operating systems or custom images
- Pay per hour of use
• Magnetic or SSD drives
- attach to running instances
- provisioned or size-dependent IOPs
- up to 64 Tb (SSD)
- backups
• Networking & Firewall
- Easy IP management
- Load balancers
- VPN & VPC
9. Bootstrap your application
var express = require('express');
var bodyParser = require('body-parser');
var shortHash = require('short-hash');
var LocalStorage = LocalStorage;
var storage = new (require('node-localstorage').LocalStorage)('./storage');
var app = express();
app.use(express.static(__dirname + '/public'));
app.use(bodyParser.json());
app.get('/', function (req, res) {
res.render('index.html')
});
10. Add your shorten logic
app.post('/shorten', function (req, res) {
var url = req.body && req.body.url;
if (!url) {
return res.status(400).json({message: 'no URL'}).end();
} else {
var hash = shortHash(url);
storage.setItem(hash, url);
res.json({
hash: hash,
shortUrl: process.env.HOST + hash,
targetUrl: url});
}
})
11. Add your redirection logic
app.get('/:key', function (req, res) {
var url = storage.getItem(req.params.key);
if (url) {
res.redirect(302, url);
} else {
res.status(404).send('Sorry, link not found');
}
});
var server = app.listen(process.env.PORT || 8888);
16. Elastic Beanstalk
• Quick deployment and management of Web application
• PHP, ASP.NET, Python, NodeJS, Docker etc
• No need to setup infrastructure
• Performance metrics
• Scale up and scale down
• Rolling updates
22. Traditional SQL databases hosteb my Amazon - RDS
• Fully managed traditional SQL servers
- MySQL
- MS SQL
- PostgreSQL
- Oracle
• Variety of server sizes
• Licenses included into price
• Automated backups
• Restore to point at a time
23. • Document DB
• No limits on size
• Provisioned throughput
• Differents attribute types including Lists and JSON
DynamoDB
24. Get DynamoDB high level API
npm install aws-sdk --save
var AWS = require("aws-sdk");
AWS.config.update({region: 'eu-west-1'});
var db = new AWS.DynamoDB.DocumentClient(); // <!-- high level SDK
25. Put hash to DynamoDB
app.post('/shorten', function (req, res) {
var url = req.body && req.body.url;
…
var hash = shortHash(url);
db.put({
TableName: "short_links",
Item: {"hash": hash, targetUrl: url}
}, function () {
storage.setItem(hash, url);
res.json(...);
})
…
26. Get from DynamoDB
var hash = req.params.key;
var url = storage.getItem(hash);
if (url) {
return go(res, url);
}
db.get(
{TableName: "short_links", Key: {"hash": hash}},
function (err, record) {
url = record.targetUrl;
url && storage.setItem(hash, url); //local cache
go(res, url);
});
27. Security considerations - IAM
• Identity and Access Management
• Generate policies that describes
- what operations
- are allowed or denied
- to what AWS resources
- (in what circumstances)
• Apply policy to:
- IAM User identified by Access Key and Secret key
- EC2 instance
• vim ~/.aws/credentials
[dev]
aws_access_key_id = AKIA....
aws_secret_access_key = PQC+9o…
• AWS_PROFIEL=dev PORT=8000 HOST=http://localhost:8000/ npm run start
29. Adding screenshot functionality
var webshot = require('webshot');
webshot(targetUrl, hash + '.png', function(err) {
// Screenshot is ready. Where I should save it?
});
34. Screenshotting is just a function. Lambda in on the stage
• Invoke code in response to events
- File uploads to S3
- DynamoDB table changes
- SNS notifications
- HTTP requests
- Scheduled executions
• Just bring your code
- Node.JS
- Python 2.7
- Java 8
• Pay per invokations and computation power
- making 5M screenshots will cost you ~$42*
* 512 Mb RAM, 2 minutes timeout
35. Screenshotting function with writing results to DynamoDB
exports.handler = function (data, context) {
var hash = …;
var targetUrl = …;
makeScreenshot(hash, targetUrl, function(err, r) {
if (err) {return context.fail(err)};
db.put({
TableName: "short_links",
Item: {"hash": hash, screenshotUrl: r.Location}
}, context.done);
})
};
36. Hookup Lambda to new shorten links. DynamoDB streams is on the stage
Ordered stream of changes happened to DynamoDB table
• Any change in table is recorded
• Old record and new record can be stored together with change
• Multiple applications processing stream
• Lambda can be easily attached
37. Fine tuning screenshotting Lambda
exports.handler = function (data, context) {
var item = data.Records[0].dynamodb.NewImage;
var hash = item.hash.S;
var targetUrl = item.targetUrl.S;
var screenshotUrl = item.screenshotUrl.S;
if (screenshotUrl){
return context.success("Screenshot already taken");
}
makeScreenshot(hash, targetUrl, function(err, r) {
...
39. Static content delivery - CloudFront
• CDN for static and dynamic content
• Web and Streaming distributions
• Seamless integration with S3
• Own SSL certificates, free for SNI
• Requests logging and delivery to S3
• Serving dynamic content from Beanstalk, Load balancers and own servers
• Device detection
• Cost effective
48. Too many setup were done manually
CloudFormation
• describe services you need
• specify policies
• use configuration variable
• and let Amazon handle it
49. Quick recap
• Beanstalk - websites hostings
• RDS - hosted relational DB
• DynamoDB - document DB with cool features
• SQS - simple queue service
• S3 - infinite file storage
• CloudFront - static and dynamic content delivery
• Lambda - running code in response to event
• Kinesis - operate with data streams as wizard
• IAM - security in cloud
• CloudFormation - automated cloud resources provisioning