2017年のLINEのマイクロサービスを支えるSpring

2017年の
LINEのマイクロサービスを
⽀えるSpring
Masahiro Ide, LINE Corporation

@imasahiro
https://github.com/imasahiro
Self Intro
l Masahiro Ide
l Server side engineer at LINE
l Mainly worked on スタンプ/着せかえショップ
l Contribute to Armeria - Asynchronous RPC/REST library
l https://github.com/line/armeria

• Where is Spring used at LINE?
• Useful/Customized Spring features
• Monitoring
Agenda

LINE System
l LINE has hundreds of applications
l Recent apps are Java/Spring applications
l Mainly communicates with Thrift, and REST
l Uses same in-house deployment, monitoring system

今⽇お話しするLINEのシステム
lスタンプショップ

スタンプショップ
l Spring-boot 1.5.8, Spring-MVC
l Mostly Asynchronous Http/2 Thrift/REST service
l Java8, RxJava2, (Completable|Listenable)Future
l Storage
l Redis, MySQL, Mongo, ElasticSearch
l Monitoring
l Micrometer, Dropwizard Metrics, Prometheus, Zipkin

Sticker server Mongo
MySQL
ElasticSearch
Redis
LINE Client
Theme server Search server
STORE Server
スタンプショップ構成

l Easy to setup
l Have many integrations with other libraries
l Mybatis, Thymeleaf, Tomcat, etc.
Why spring, why spring-boot?
@SpringBootApplication
public class SampleApp {
@Configuration
class MvcConf extends WebMvcConfigurerAdapter {…}
public static void main(String[] args) {
SpringApplication.run(SampleApp.class, args);
}
}

Simple & Useful Spring features
1. Spring cache abstraction
1. (Spring) AOP
2. Dependency management

1. Spring Cache Abstraction
l SpringにおけるCacheの抽象化
l Can choose cache implementation
l Redis, Caffeine, Guava, etc.
l …But need to use carefully on async service
@Cacheable(cacheNames = "listCache")
List<Item> selectByLang(String lang) {
return sqlSession.selectList(…);
}

Async support in cache abstraction
lNaive (and wrong) solution
l This just caches a Future…
l But donʼt cache value in Future
l See https://jira.spring.io/browse/SPR-12967
@Cacheable(cacheNames = "listCache")
CompletableFuture<List<Item>> selectByLang(String lang) {
return CompletableFuture.supplyAsync(
() -> sqlSession.selectList(…), executor);
}

Async support in cache abstraction
lMade similar cache abstraction
l with Caffeine AsyncLoadingCache and
Lettuce (Async redis client)
@AsyncCacheable(cacheNames = "listCache")
CompletableFuture<List<Item>> selectByLang(String lang) {
() -> sqlSession.selectList(…), executor);
}

2. (Spring) AOP
lAOP - Aspect Oriented Programming
l Use case: logging, perf instrumentation
@Loggable @GetMapping(“/”)
ModelAndView home(…) {…}
@Around("@annotation(Loggable)")
public Object logging(ProceedingJoinPoint pjp) {
Timer timer = newTimer()
try {
return pjp.proceed();
} finally {
long latency = timer.stop();
logger.info(“request path: {}, args:{}, latency:{}”, …, latency));
}
}

Client side MySQL sharding using AOP
ApplicationApplication
MySQL Slave
MySQL Master
Shard01
MySQL Slave
MySQL Master
Shard01
MySQL Slave
MySQL Master
Shard01
@Shardable
List<Item> selectListByUserId(int id) {…}
List<DataSource> dataSources = …;
@Around("@annotation(Shardable)")
Object sharding(ProceedingJoinPoint pjp) {
int id = getArguments(pjp)[0];
source = dataSrouces.get(id % 12);
useDataSource(source);
return pjp.proceed();
}

3. Dependency management
lBOM and Dependency management plugin
lSpringがlibrary versionを管理してくれる
dependencyManagement {
imports {
mavenBom 'io.spring.platform:platform-bom:Brussels-SR5'
}
dependencies {
dependencySet(group: 'com.linecorp.armeria', version: '0.55.0') {
entry 'armeria-tomcat-shaded’
entry 'armeria-thrift0.9-shaded'
}
dependency 'biz.paluch.redis:lettuce:4.4.1.Final’
…
}}

How to make a service asynchronous?
l元々はTomcat+Spring MVC or thrift+DBなシステム
@Repository public class ShopStorage {
return sqlSession.selectList(
“shop.selectByLang”, lang);
}}
@Controller public class ShopController {
@ResponseBody List<Item> home() {
return storage.getList();
}}

How to make a service asynchronous?
lSync accessを１つずつAsyncに書き直す
@Repository public class ShopStorage {
return sqlSession.selectList(
“shop.selectByLang”, lang);
}}
@Controller public class ShopController {
@ResponseBody List<Item> home() {
return storage.getList();
}}
@Repository public class AsyncShopStorage {
CompletableFuture<List<Item>>
selectByLang(String lang) {
() -> sqlSession.selectList(
“shop.selectByLang”, lang),
executor);
}}
@Controller public class AsyncShopController {
@ResponseBody
DeferredResult<List<Item>> home() {
result = new DeferredResult<>();
storage.getList()
.thenApply(result::setResult);
return result;
}}

How to observe our services?
lLog
l その瞬間に何が起こったかを確認する
lMetric
l 現在起きている現象のトレンドを確認する
lTrace
l システム間で何が何が起きているかを確認する

How to observe our services?
lLog
l Slf4j + Kibana
lMetric
l Micormeter, Prometheus, Dropwizard metrics
l Grafana
lTrace
l Zipkin
l io.zipkin.brave:brave-instrumentation-mysql
l io.zipkin.brave:brave-instrumentation-spring-webmvc
l And elasticsearch, mongo, redis integrations, etc.

Micrometer
l Supports multiple monitoring tools
l Dropwizard Metrics, Prometheus, etc.
l Will be provide as a Spring Boot 2.0 metrics collection
l Already used in production servers
l Found some concurrency issues but now looks stable
l https://github.com/micrometer-metrics/micrometer/issues/139

Prometheus
lSimple TSDB, Pull型
lSupports querying, alerting
Application
# HELP armeria_client_response_duration_seconds
# TYPE armeria_client_response_duration_seconds summary
armeria_client_response_duration_seconds{method="sendMessage",
service=”Service",quantile="0.5",} 8.194E-4
…
/internal/metrics
Node exporterPrometheus
pull ApplicationNode exporter

Put all together
lArmeria - Spring, micrometer, zipkin integration
@Bean
ThriftServiceRegistrationBean service(MyService.AsyncIface myService,
Tracing tracing) {
return new ThriftServiceRegistrationBean()
.setPath("/thrift/")
.setService(
THttpService.of(shopCapabilityService)
.decorate(LoggingService.newDecorator())
.decorate(MetricCollectingService.newDecorator(
MeterIdFunction.ofDefault("LineShopService")))
.decorate(HttpTracingService.newDecorator(tracing))));
}

2017年のLINEのマイクロサービスを支えるSpring

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à 2017年のLINEのマイクロサービスを支えるSpring

Similaire à 2017年のLINEのマイクロサービスを支えるSpring (20)

Plus de LINE Corporation

Plus de LINE Corporation (20)

Dernier

Dernier (20)

2017年のLINEのマイクロサービスを支えるSpring