Seastar Summit 2019 Keynote

1. Seastar Summit 2019
Keynote
Avi Kivity, maintainer
@AviKivity

2. What is Seastar?
● Treat multicore and disk I/O as asynchronous entities, like networking
○ Locks and atomic operations replaced by message passing
○ Synchronous disk I/O replaced by AIO
● Exploit message passing for performance
○ Look ma, no locks
○ Anywhere
○ Deep NUMA binding
● Bypass page cache, kernel networking* to avoid same problems in the kernel
● Wrap everything with composable APIs
○ Everything returns a future

3. What is Seastar?
● Stop pretending
○ That we can allocate any amount of memory we want
○ That we can run any number of threads we want
○ That we can throw any amount of I/O at the system and get good latency
● Start scheduling
○ A memory allocation is coupled with a memory reclaim, so application must schedule its
memory allocations
○ Threads, tasks, and I/O are scheduled by Seastar, so the application has a say in what is
going on at any time

4. What is Seastar?
● Deep investment by application writers
○ Do everything the Seastar way
○ Glue or convert third party libraries
○ Wait for long compile times
○ Wait for new language features
● Deep rewards from Seastar
○ High throughput
○ Low latency
○ Great control of where the throughput and latency go
■ Service level agreements
■ Multi-tenancy

5. What is Seastar?
● A framework for disk-network-compute-multicore intensive applications
○ Databases
○ Filesystems
○ Message queues

6. What is Seastar?
● What we want it to be

7. A Brief History
commit 193ac5919df51df2366604fc5cbfd73d15bb52f5
Author: Avi Kivity <avi@cloudius-systems.com>
Date: Sun Aug 10 08:56:33 2014 +0300
Initial commit
reactor.hh | 168 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
sstring.hh | 124 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
main.cc | 18 ++++++++++++++++++
reactor.cc | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
test-reactor.cc | 29 +++++++++++++++++++++++++++++
Makefile | 16 ++++++++++++++++
6 files changed, 440 insertions(+)

8. First non-Avi commit
commit 34667e8dfe599488448435e454a7e413d863c421
Author: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Date: Wed Aug 27 13:28:23 2014 +0200
reactor: fix io_setup() failure
_io_context must be set to 0, otherwise io_setup fails with -EINVAL.
Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
reactor.cc | 1 +
1 file changed, 1 insertion(+)

9. First non-Cloudius commit
commit bf564b463a2ad252a024f717c3d43e646c48867d
Author: Marek Waszkiewicz <marek.waszkiewicz77@gmail.com>
Date: Thu Mar 26 01:16:23 2015 +0100
README: add xen-devel boost-devel for fedora 21 build instructions
Signed-off-by: Marek Waszkiewicz <marek.waszkiewicz77@gmail.com>
README.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

10. Non-Scylla users start appearing
commit 8f6dee7c7b2db68060c32c3bada2a4d420b92ae0
Author: Alexander Gallego <gallego.alexx@gmail.com>
Date: Mon Jan 18 10:58:24 2016 -0500
Initial commit
.gitignore | 28 ++++++++++++++++++++++++++++
LICENSE | 201 ++++++++++++++++++++++++++++++++++++
README.md | 2 ++
3 files changed, 231 insertions(+)
commit 85b4a44546462e3a2e2451bb4fea5befebaa10a9
Author: dudes-come <dudes@qq.com>
Date: Mon Dec 18 10:17:16 2017 +0800
add readme and license
LICENSE | 7 +++++++
README.md | 11 +++++++++++
2 files changed, 18 insertions(+)
commit a467e5b1cf2e7e0fe2986f4a23e91a37c9ab98b4
Author: Casey Bodley <cbodley@redhat.com>
Date: Sat Jan 27 12:24:12 2018 -0500
cmake: skip boost dependency on ALIAS targets
the boost override for add_library() does not work for ALIAS targets:
CMake Error at cmake/modules/BuildBoost.cmake:227 (add_dependencies):
Cannot add target-level dependencies to alias target "Seastar::seastar".

11. Notable APIs
● Execution stages
○ Mitigating instruction cache footprint
● Schedulers
○ If you have >1 user
● Alien
○ Providing a migration path for existing applications
● Stream/subscription*
○ More structured computations

12. Challenges
● Documentation, ramping up
● Destructors and the loss of RAII
● Object lifecycle control
● Task and allocation proliferation
● Instruction cache footprint

13. Opportunities
● Coroutines
○ Clean up code
○ Reduce task allocations
○ Reduce task executions
○ Maybe help w/ destructors
○ Coolness factor eπ!

14. Modules
● Reduce compilation time and object bloat

15. Stream/Subscription Revamp
● Memory accounting, not fixed size queue
● Use co_yield to generate data, co_await to consume data
● Avoid producer/consumer interlocks
● Avoid overruns

16. Task co-execution
● Automatic execution stage for every task type
○ Reduce instruction cache footprint
○ Need extensive testing

17. Nested scheduling
● Improve clarity of scheduling model
○ Following in Linux cgroups footsteps

18. Compatibility statements
● No ABI compatibility outside external interfaces
● Source compatibility w/ deprecation periods
○ Deprecation period can vary depending on feature
○ Examples: variadic future, sever_socket::accept()
● 2 Standard revisions
○ C++14, C++17 -> C++17, C++20
● 2½ Compiler revisions
○ gcc 8, gcc9
○ When gcc 10 is released, half a year grace period for gcc 8
● 2 Compiler vendors
● 2 networks stacks
● All contributed architectures

19. Compatibility statements
● Compatibility is community-driven
○ If you want a compiler supported, run a continuous integration bot
○ If you want source compatibility for an API, contribute fixes and unit tests
○ If you want an architecture supported, run a continuous integration bot
● Maintainers are there to help and enforce, not fix the bugs

20. while (true) {
auto q = co_await question();
co_await answer(std::move(q));
}
co_return thanks();