Do you need to run an AWS Lambda function on a schedule, without an event to trigger the invocation? This session shows how to use an Amazon CloudWatch metric and CloudWatch alarms, Amazon SNS, and Lambda so that Lambda triggers itself every minute—no external services required! From here, other Lambda jobs can be scheduled in crontab-like format, giving minute-level resolution to your Lambda scheduled tasks. During the session, we build this functionality up from scratch with a Lambda function, CloudWatch metric and alarms, sample triggers, and tasks.

1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Guy Davies, Sophos Ltd.
October 2015
CMP407
Lambda as Cron
Scheduling Invocations in AWS Lambda

2. About me
• Guy Davies, Senior Systems Engineer, Sophos Ltd
Sophos use AWS extensively in our operations to support our anti-
malware, anti-spam, and threat-detection software and hardware
devices.
Also provide security products for AWS customers to complement their
cloud security profiles:
• UTM [ free trial available! ]
• Secure Server for Linux [ AMI available in Marketplace ]
www.sophos.com/aws

3. What to expect from the session
• How to schedule tasks in AWS Lambda
• Overview of the various options available
• Building up a pure Lambda scheduling infrastructure
• Resources and templates to implement this yourself

4. Events? Scheduling?

5. Why would I want to schedule Lambda?
• Lambda designed as event-driven computing
• Event-driven computing is awesome
• Most of the time you want to trigger because something
happens…
HOWEVER

6. Why would I want to schedule Lambda?
Sometimes you just plain need to do something on a
schedule.
Examples:
• Log cleanup
• Batching up statistics
• Alarm clock
• Infrastructure automation

7. Scheduling options on traditional infrastructure
Unix
• cron: recurring tasks
*/2 * * * * do_something
• at: run a task at a specific time
at 1615 oct 7 <<< “mail –s’AWS talk!’ guy.davies@sophos.com”
Windows
• Scheduled tasks
• AT

8. Options for scheduling Lambda functions (1)
1. Spin up an Amazon EC2 instance, and use crontab to
invoke Lambda
• Why bother running Lambda at all?
• Many folks want a pure-Lambda deployment
• Running an instance means more to manage
• Not hugely financially efficient

9. Options for scheduling Lambda functions (2)
2. Unreliable Town Clock (townclock.io)
• Awesome public Amazon SNS topic
• Chimes every fifteen minutes
• Community supported
• Has a 15 minute granularity

10. Options for scheduling Lambda functions (3)
3. Others…
• Trigger from Amazon SWF
• Trigger from an instance in AWS Data Pipeline
• Trigger from an AWS CloudTrail upload into an Amazon S3
bucket
All of these could be your solution! But what if we want a
“pure” Lambda implementation that’s managed by AWS?

11. A pure Lambda scheduler

12. How do we generate a timing signal in AWS?
0
0.2
0.4
0.6
0.8
1
Photo: ‘Signetics NE5555N, …’ by Stefan506 is
licenced by CC BY-SA 3.0. Source:
https://en.wikipedia.org/wiki/555_timer_IC
Amazon
CloudWatch

13. CloudWatch as a time signal
1. Set an alarm on a CloudWatch metric
2. Alarm goes into ALARM state:
• Triggers Amazon SNS which triggers Lambda
• Lambda inverts the state of the metric
3. Alarm goes into OK state:
• Triggers SNS which triggers Lambda
• Lambda inverts the state of the metric

14. Configuring the CloudWatch alarm

15. Configuring the CloudWatch alarm
Once the metric is
inverted, the alarm will
trigger at the top of the
next minute:
1-minute resolution

16. Configuring the CloudWatch alarm
All three states trigger the
same SNS notification. The
Lambda function figures
which was the trigger and
sets the CloudWatch
metric to the opposite state

17. Putting it together
Lambda
cron
function
CloudWatch
metric
CloudWatch alarm
triggers
SNS topic10
Invoke further Lambda
functions (if scheduled)

18. The Lambda function

19. Lambda function
1. The “event” is the SNS notification that is sent by
CloudWatch.
2. Invert the value of the CloudWatch metric.
3. Jobs and schedule are managed as a separate JSON
file in the bundle.
4. Invoke any Lambda functions in the schedule that are
scheduled to be run this minute.
5. Done.

20. Main Lambda function
exports.handler = function(event, context) {
async.waterfall([
function (callback) { flip_cloudwatch(event,callback); },
read_crontab,
execute_lambdas
], function (err) {
if (err) context.fail(err);
else context.succeed(); });
};

21. Flipping CloudWatch
The event is an SNS message from CloudWatch:
var snsmessage = JSON.parse(event.Records[0].Sns.Message);
If it’s just gone into alarm, we want to reset to zero. Likewise if it’s just gone into
OK, reset to 1.
if (snsmessage.NewStateValue == 'ALARM') { value = 0.0 }
else if (snsmessage.NewStateValue == 'OK' || snsmessage.NewStateValue ==
'INSUFFICIENT_DATA') { value = 1.0 };
Push the new value to CloudWatch:
var params = { MetricData: [ { MetricName: 'LambdaCron', Timestamp: new Date,
Unit: 'None', Value: value } ], Namespace: 'LambdaCron' };
cloudwatch.putMetricData(params, function(err, data) { . . . });

22. Crontab-like Lambda configuration
{
"jobs": [ {
"schedule": "*/3 * * * *",
"function": "testfunction",
"args": {
"key1": "test1",
"key3": "test3",
"key2": "test2"
}
} ]
}

23. Checking the schedule (1)
Use the fantastic cron-parser library (https://github.com/harrisiirak/cron-parser/
MIT licenced)
var parser = require('cron-parser');
Create a Date object which refers to the top of the current minute to compare
the schedule to.
var d = new Date();
d.setSeconds(0);
d.setMilliseconds(0);
Parse the crontab to find the ‘next’ runtime for the job (= now if it needs to run)
var interval =
parser.parseExpression(job["schedule"],{currentDate: d});
var runtime = interval.next();

24. Run each job that needs running
if (datestring == runtimestring) {
var lambda = new AWS.Lambda();
var params = {
FunctionName: job["function"],
InvocationType: "Event",
Payload: JSON.stringify(job["args"])
};
lambda.invoke(params, function(err,data) {
if (err) iteratorcallback(err);
else iteratorcallback(null);
});
}
}

25. Demo

26. Use cases
Trialling this for intelligent scaling on a schedule
• AWS scheduled scaling has limitations:
• Absolute number of desired instances
• Limit of 120 tasks per month
• Lambda function triggered by lambda-cron could do for
example:
• Every morning at 08:00 UTC, add 20% of capacity unless we
have > 30 instances already running

27. Reliability and monitoring
Empirically pretty reliable.
Running since April without (much!) intervention.
Even comes back after outages!

28. Reliability and monitoring
Monitoring: CloudWatch!
• Lambda invocation metrics will tell you that it’s running
• Application-level monitoring on the jobs you are
triggering
60 invocations / hr

29. Summary
1. Use CloudWatch alarms as two states to provide a
timing signal to Lambda.
2. Trigger off all three states to enhance reliability.
3. Allows us to schedule tasks purely within Lambda.
4. The cron function invokes once a minute.

30. Resources
Github:
github.com/g-a-d/lambda-cron
Email:
guy.davies@sophos.com
AWS
CloudFormation
stack
Lambda
function

31. Resources
Libraries used:
• async: https://github.com/caolan/async
• avoid nested-callback-hell (great for folks more used to
procedural programming)
• MIT license
• cron-parser: https://github.com/harrisiirak/cron-parser
• parse crontabs
• MIT license

32. Resources
Remember to check www.sophos.com/aws
• UTM for AWS (free trial available)
• Secure server for AWS
• Free trials, free home use AV

33. Remember to complete
your evaluations!

34. Thank you!