At Clever Cloud, we are working on extremely light virtual machines to run WebAssembly binaries. As it’s WASM, we can write code using a lot of languages. We use a custom unikernel to run this WASM as Function-as-a-Service, using one VM per function execution. These VM can run on events from messages coming through Pulsar, or from HTTP invocation, the run is on-demand as only the consumers stay up. This can be a new model: Pulsar functions for real isolation in multi-tenancy use cases. This talk will show the use case, explain the virtualization underneath and demonstrate the multi-tenancy use case.

1. Building a Function as a Service with Pulsar

2. Alexandre DUVAL
@kannarfr
IT Engineer @CleverCloud
Logs systems
Automatic Add-ons migration systems
Stuff around Biscuit
Pulsar Add-on (alpha through support)
PaaS On-Call human (not today :D)

3. Geoffroy COUPRIE
@gcouprie
Security Engineer @CleverCloud
Sozu HTTP reverse proxy
Function as a Service
Biscuit authorization tokens
Working on the Pulsar Rust client 1.0

4. Function as a Service

5. Function as a Service?
- Run code on demand
1@gcouprie

6. Function as a Service?
- Run code on demand
- Without caring about keeping infrastructure up
1@gcouprie

7. Function as a Service?
- Run code on demand
- Without caring about keeping infrastructure up
- Or scaling
1@gcouprie

8. Function as a Service?
- Run code on demand
- Without caring about keeping infrastructure up
- Or scaling
- Billed on consumption
1@gcouprie

9. FaaS by Clever Cloud
- Build your functions in a language that can generate Web Assembly
2@gcouprie

10. FaaS by Clever Cloud
- Build your functions in a language that can generate Web Assembly
- That will be compiled to machine code
2@gcouprie

11. FaaS by Clever Cloud
- Build your functions in a language that can generate Web Assembly
- That will be compiled to machine code
- Loaded and executed in ephemeral virtual machines without operating
systems
2@gcouprie

12. FaaS by Clever Cloud
- Build your functions in a language that can generate Web Assembly
- That will be compiled to machine code
- Loaded and executed in ephemeral virtual machines without operating
systems
- That boot on the fly for each request or message on a topic
2@gcouprie

13. Web Assembly
- Good compilation target (supported by various languages)
3@gcouprie

14. Web Assembly
- Good compilation target (supported by various languages)
- Designed for sandboxing
3@gcouprie

15. Web Assembly
- Good compilation target (supported by various languages)
- Designed for sandboxing
- Not limited to browsers!
3@gcouprie

16. One Virtual Machine per request
- Virtual machines are cheap: 1 to 10MB RAM
4@gcouprie

17. One Virtual Machine per request
- Virtual machines are cheap: 1 to 10MB RAM
- Fast: boot in 1ms
4@gcouprie

18. One Virtual Machine per request
- Virtual machines are cheap: 1 to 10MB RAM
- Fast: boot in 1ms
- Great isolation: very limited host API, no hardware emulation
4@gcouprie

19. Infrastructure based on Pulsar
- Compilation service gets compilation requests from a topic
5@gcouprie

20. Infrastructure based on Pulsar
- Compilation service gets compilation requests from a topic
- FaaS host gets configuration information (new apps, new code) from a topic
5@gcouprie

21. Infrastructure based on Pulsar
- Compilation service gets compilation requests from a topic
- FaaS host gets configuration information (new apps, new code) from a topic
- Metrics and logs are sent via another topic
5@gcouprie

22. Pulsar Functions

23. Building Pulsar Functions
- No need to start one process per function
6@gcouprie

24. Building Pulsar Functions
- No need to start one process per function
- The Faas host has 1000s of consumers
6@gcouprie

25. Building Pulsar Functions
- No need to start one process per function
- The FaaS host has 1000s of consumers
- Starts a function when a message arrives on a topic
6@gcouprie

26. Multi-Tenancy Challenges
- Functions can subscribe to topics from various clients
7@gcouprie

27. Multi-Tenancy Challenges
- Functions can subscribe to topics from various clients
- But there’s only one authentication context per connection
7@gcouprie

28. Multi-Tenancy Challenges
- Functions can subscribe to topics from various clients
- But there’s only one authentication context per connection
- Should we have one TCP connection per consumer?
7@gcouprie

29. Biscuit

30. Biscuit
New authentication & authorization token
Specifications
github.com/CleverCloud/biscuit
Java Library
mvnrepository.com/artifact/com.clever-cloud/biscuit-java
Rust Crate
crates.io/crates/biscuit-auth
8@kannarfr

31. Biscuit
New authentication & authorization token
Powerful authorization rules
Based on a Datalog variant
facts, rules, caveats
8@kannarfr

32. Biscuit
New authentication & authorization token
Powerful authorization rules
Decentralized
Verification & attenuation are
decentralized
8@kannarfr

33. Biscuit
New authentication & authorization token
Powerful authorization rules
Decentralized
Biscuit-Pulsar plugins!
Biscuit-Pulsar authorization plugin
github.com/CleverCloud/biscuit-pulsar
8@kannarfr

34. Biscuit Example Applied to Pulsar

35. - Start from a token with full access to a namespace
Attenuation
9@kannarfr

36. - Start from a token with full access to a namespace
- A team needs Pulsar access: reduce access to lookup/produce/consume on a
topic name prefix
Attenuation
9@kannarfr

37. - Start from a token with full access to a namespace
- A team needs Pulsar access: reduce access to lookup/produce/consume on a
topic name prefix
- Application specific token: reduce team token to subscribe only, on a fixed
topic, with a fixed subscription name
Attenuation
9@kannarfr

38. Authorization life cycle on FaaS

39. - A function comes with its Biscuit token
Authorizing Pulsar Function
10@gcouprie

40. - A function comes with its Biscuit token
- The FaaS host can authorize the token with the same rules as the
Biscuit-Pulsar plugin
Authorizing Pulsar Function
10@gcouprie

41. Authorizing Pulsar Function
- A function comes with its Biscuit token
- The FaaS host can authorize the token with the same rules as the
Biscuit-Pulsar plugin
- The FaaS host subscribes and produces with only one authentication token,
on a reduced number of connections
10@gcouprie

42. Needs in Pulsar

43. Biscuit Pulsar Requirements
Boolean allowNamespacePolicyOperation(NamespaceName namespaceName,
PolicyName policy,
PolicyOperation operation,
String originalRole,
String role,
AuthenticationDataSource authData) {}
Issue #5720: fixed.
Put granularity in Pulsar’s AuthorizationService
11@kannarfr

44. Pulsar Needs: wishlist
- PIP-51: tenant policy support (and topics)
12@kannarfr

45. - PIP-51: tenant policy support (and topics)
- Pulsar Proxy for protocols handlers
Pulsar Needs: wishlist
12@kannarfr

46. - PIP-51: tenant policy support (and topics)
- Pulsar Proxy for protocols handlers
- Topic compaction on the fly
Pulsar Needs: wishlist
12@kannarfr

47. - PIP-51: tenant policy support (and topics)
- Pulsar Proxy for protocols handlers
- Topic compaction on the fly
- Pattern implementation as Exchanges, FanOuts, … at broker level
Pulsar Needs: wishlist
12@kannarfr

48. - PIP-51: tenant policy support (and topics)
- Pulsar Proxy for protocols handlers
- Topic compaction on the fly
- Pattern implementation as Exchanges, FanOuts, … at broker level
- Metrics: enable pulsar to send its metrics itself to as timeseries
Pulsar Needs: wishlist
12@kannarfr

49. clever-cloud.com

50. Annex 1: Biscuit Example Applied to Pulsar

51. authority: Block[0] {
symbols: [admin]
context:
facts: [
revocation_id("af136428-b9c3-4360-8535-3a147ce80d99"), // good practice
right(#authority, #admin)
]
rules: []
caveats: []
}
Authority (block 0)
Facts are basically data used by verifier.

52. blocks: [
Block[1] {
symbols: [limited_right, topic_operation]
context:
facts: []
rules: []
caveats: [
*limited_right("tenantTest", "namespaceTest", $2, $3) <-
topic_operation(#ambient, "tenantTest", "namespaceTest", $2, $3)
]
}
]
Attenuated (block 1)
$2 is a slot for topic name, $3 for consume, producer & lookup

53. blocks: [
Block[1] {
symbols: [limited_right, topic_operation, namespace_operation]
context:
facts: []
rules: []
caveats: [
*limited_right("tenantTest", "namespaceTest", $2, $3) <-
topic_operation(#ambient, "tenantTest", "namespaceTest", $2, $3)
|| (OR)
*limited_right("tenantTest", "namespaceTest", $2) <-
namespace_operation(#ambient, "tenantTest", "namespaceTest", $2)
]
}
]
Attenuated (block 1)
Here $2 is a slot for namespace operation (#offload_write)

54. Verifier built from Biscuit token
facts: [
revocation_id("af136428-b9c3-4360-8535-3a147ce80d99"),
right(#authority, #admin)
],
rules: [],
caveats: [
*limited_right("tenantTest", "namespaceTest", $2) <-
namespace_operation(#ambient, "tenantTest", "namespaceTest", $2)
// || topic_operation query which is unused here is the final example.
]
Verifier facts, rules, caveats are computed from biscuit token.

55. Verifier add contextual facts
facts: [
revocation_id("af136428-b9c3-4360-8535-3a147ce80d99"),
right(#authority, #admin),
namespace(#ambient, "tenantTest", "namespaceTest")
namespace_operation(#ambient, "tenantTest", "namespaceTest", #offload_write)
],
rules: [],
caveats: [
*limited_right("tenantTest", "namespaceTest", $2) <-
namespace_operation(#ambient, "tenantTest", "namespaceTest", $2)
// || topic_operation query which is unused here is the final example.
]
Then authorization check adds its context.

56. Verifier add rules & caveats
facts: [
revocation_id("af136428-b9c3-4360-8535-3a147ce80d99"),
right(#authority, #admin),
namespace(#ambient, "tenantTest", "namespaceTest"),
namespace_operation(#ambient, "tenantTest", "namespaceTest", #offload_write)
],
rules: [
*right(#authority, $0, $1, $2) <-
right(#authority, #admin),
namespace_operation(#ambient, $0, $1, $2) @ $2 in [offload_write, ...]
],
caveats: [
*limited_right("tenantTest", "namespaceTest", $2) <-
namespace_operation(#ambient, "tenantTest", "namespaceTest", $2),
// || topic_operation query which is unused here is the final example.
*authorized() <- (#authority, $0, $1, $2),
namespace_operation(#ambient, $0, $1, $2) @ $2 in [offload_write, ...]
]

57. Verify life cycle
- Add biscuit token facts & verifier’s facts (ambient ones)
- Add rules from biscuit and verifier
- Generate all the facts
- Ensure that token and verifier’s caveats pass and authorize