3. Agenda
• What is ORM
• Pros / Cons of JPA
• Current status in Coupang
• Alternatives of JPA
• Slick
• jOOQ
• Exposed
• Requery
4. ORM (Object Relational Mapping)
• OOP’s object graph vs Relational Database
• Focus OOP, not Relational Database
• No matter of RDBMS vendor - Same code
• Hibernate coverage is 95% over traditional SQL statements
• ORM not suitable for data centric application in performance
• Why should you use an ORM?
5. Pros of JPA
• Focus to Java Object graph & OOP
• No need to know relations and constraints of entities
• No need to know specific DB features by various vendor
• No need to know SQL, just use Java API
• Supplement by HQL or JPQL or QueryDSL
• All support for Stateful, Stateless (Default is Stateful)
6. Cons of JPA
• If you knew SQL already, JPA is wired
• Hard to learning (exponential)
• Low performance by stateful and fetch by id
• No suitable for Bulk or Set operations
• Massive Insert, Statistical Summary (Cube …)
• Non-Threadsafe Session - Low throughput
• Need to learn specific JPAVendor (Hibernate, Eclipse Link)
• HQL, @DynamicInsert, @LazyCollection
• 2nd Cache (recommend JCache (JSR-305))
7. Current Status in Coupang
• No Deep Dive
• Missing override (hashCode, equals, toString)
• No using @NatualId
• Bad Policy / Bad Design
• Every entity has Own identifier (Some case no need)
• Poor performance -> Mislead “JPA is bad”
• Apply not suitable case
• Bulk operations, Statistical operations
• No use supplement features
• StatelessSession, 2nd Cache …
9. Features in Alternatives of JPA
• Design Principle
• OOP based, Support Multi DBVendor
• No need stateful for Reference object
• Support association, inheritance, converter in JPA
• Performance
• Speed up to Plain SQL
• Stateless
• Support Asynchronous or Reactive
• Support Bulk or Batch operations
18. jOOQ
• Reflect Database Schema to generate Entity Class
• Typesfe SQL (akaTypesafe MyBatis)
• Database First (Not ORM)
• Stateless
• Need DBMS Owner Authority
• jOOQ vs Hibernate :When to choose which
19. jOOQ - typesafe SQL
SELECT * FROM BOOK
WHERE BOOK.PUBLISHED_IN = 2011
ORDER BY BOOK.TITLE
create.selectFrom(BOOK)
.where(BOOK.PUBLISHED_IN.eq(2011))
.orderBy(BOOK.TITLE)
select().from(t).where(t.a.eq(select(t2.x).from(t2));
// Type-check here: ---------------> ^^^^
select().from(t).where(t.a.eq(any(select(t2.x).from(t2)));
// Type-check here: -------------------> ^^^^
select().from(t).where(t.a.in(select(t2.x).from(t2));
// Type-check here: ---------------> ^^^^
select(t1.a).from(t1).unionAll(select(t2.a).from(t2));
// Type-check here: ----------------> ^^^^
select(t1.a, t1.b).from(t1).union(select(t2.a, t2.b).from(t2));
// Type-check here: -------------------> ^^^^^^^^^^
Predicates
Set operations
21. requery
• No reflection (apt code generation) - Fast instancing
• Fast startup & performance
• Schema generation
• Blocking / Non-blocking API (Reactive with RxJava)
• Support partial object / refresh / upsert
• Custom type converter like JPA
• Compile time entity validation
• Support almost JPA annotations
22. @Entity
abstract class AbstractPerson {
@Key @Generated int id;
@Index("name_index") // table specification
String name;
@OneToMany // relationships 1:1, 1:many, many to many
Set<Phone> phoneNumbers;
@Converter(EmailToStringConverter.class)
Email email;
@PostLoad // lifecycle callbacks
void afterLoad() { updatePeopleList(); }
// getter, setters, equals & hashCode automatically generated into Person.java
}
requery - define entity
Identifier
Entity class
Converter
Listeners
23. Result<Person> query = data
.select(Person.class)
.where(Person.NAME.lower().like("b%"))
.and(Person.AGE.gt(20))
.orderBy(Person.AGE.desc())
.limit(5)
.get();
Observable<Person> observable = data
.select(Person.class)
.orderBy(Person.AGE.desc())
.get()
.observable();
requery - query
Query by Fluent API
Reactive Programming
Cold Observable
Non blocking
24. @Entity(model = "tree")
interface TreeNode {
@get:Key
@get:Generated
val id: Long
@get:Column
var name: String
@get:ManyToOne(cascade = [DELETE])
var parent: TreeNode?
@get:OneToMany(mappedBy = "parent", cascade = [SAVE, DELETE])
val children: MutableSet<TreeNode>
}
requery - self refence by Kotlin
Identifier
Entity class
1:N, N:1
Cascade
25. requery - Blob/Clob usage
class ByteArrayBlobConverter : Converter<ByteArray, Blob> {
override fun getPersistedSize(): Int? = null
override fun getPersistedType(): Class<Blob> = Blob::class.java
override fun getMappedType(): Class<ByteArray> = ByteArray::class.java
override fun convertToMapped(type: Class<out ByteArray>?, value: Blob?): ByteArray? {
return value?.binaryStream?.readBytes()
}
override fun convertToPersisted(value: ByteArray?): Blob? {
return value?.let { SerialBlob(it) }
}
}
26. requery - Blob property
@Entity(model = "kt")
interface BigModel {
@get:Key
@get:Generated
@get:Column(name = "model_id")
val id: Int
@get:Column(name = "model_name")
var name: String?
@get:Convert(value = ByteArrayBlobConverter::class)
@get:Column(name = "model_picture")
var picture: ByteArray?
}
Blob column
28. Exposed - Kotlin SQL Framework
• Lightweight SQL Library
• Provide two layers of data access
• Typesafe SQL wrapping DSL
• Lightweight Data Access Object
• See : First steps with Kotlin/Exposed
• Cons
• Not mature
29. Exposed - SQL DSL
object Users : Table() {
val id = varchar("id", 10).primaryKey() // Column<String>
val name = varchar("name", length = 50) // Column<String>
val cityId = (integer("city_id") references Cities.id).nullable() // Column<Int?>
}
object Cities : Table() {
val id = integer("id").autoIncrement().primaryKey() // Column<Int>
val name = varchar("name", 50) // Column<String>
}
31. Exposed - SQL DSL - Join
(Users innerJoin Cities)
.slice(Users.name, Cities.name)
.select {
(Users.id.eq("andrey") or Users.name.eq("Sergey")) and
Users.id.eq("sergey") and Users.cityId.eq(Cities.id)
}
.forEach {
println("${it[Users.name]} lives in ${it[Cities.name]}")
}
32. Exposed - SQL DSL - Join 2
((Cities innerJoin Users)
.slice(Cities.name, Users.id.count())
.selectAll()
.groupBy(Cities.name))
.forEach {
val cityName = it[Cities.name]
val userCount = it[Users.id.count()]
if (userCount > 0) {
println("$userCount user(s) live(s) in $cityName")
} else {
println("Nobody lives in $cityName")
}
}
33. Exposed - DAO
object Users : IntIdTable() {
val name = varchar("name", 50).index()
val city = reference("city", Cities)
val age = integer("age")
}
object Cities: IntIdTable() {
val name = varchar("name", 50)
}
Schema Definition
class User(id: EntityID<Int>) : IntEntity(id){
companion object : IntEntityClass<User>(Users)
var name by Users.name
var city by City referencedOn Users.city
var age by Users.age
}
class City(id: EntityID<Int>) : IntEntity(id) {
companion object : IntEntityClass<City>(Cities)
var name by Cities.name
val users by User referrersOn Users.city
}
Entity Definition
34. Exposed - DAO Usage
val munich = City.new {
name = "Munich"
}
User.new {
name = "a"
city = munich
age = 5
}
User.new {
name = "b"
city = munich
age = 27
}
munich.users.joinToString { it.name }
User.find { Users.age.between(18, 60) }
OneTo Many
All user’s name in Munich
35. Conclusion
• Already legacy database exists ? Use only Java
• jOOQ or requery
• Scala only ? -> Slick
• Kotlin only ? -> requery, Exposed
• No matter language? -> requery
• Need Reactive programming? ->
• requery with kotlinx-requery
• kotlinx-rxjava2-jdbc ( we will open March )
42. Define Entity - Kotlin
@Entity(model = "functional")
interface Person: Persistable {
@get:Key
@get:Generated
val id: Long
@get:Index(value = ["idx_person_name_email"])
var name: String
@get:Index(value = ["idx_person_name_email", "idx_person_email"])
var email: String
var birthday: LocalDate
@get:Column(value = "'empty'")
var description: String?
@get:Nullable
var age: Int?
@get:ForeignKey
@get:OneToOne(mappedBy = "person", cascade = [CascadeAction.DELETE, CascadeAction.SAVE])
var address: Address?
@get:OneToMany(mappedBy = "owner", cascade = [CascadeAction.DELETE, CascadeAction.SAVE])
val phoneNumbers: MutableSet<Phone>
@get:OneToMany
val phoneNumberList: MutableList<Phone>
@get:ManyToMany(mappedBy = "members")
val groups: MutableResult<Group>
@get:ManyToMany(mappedBy = "owners")
val ownedGroups: MutableResult<Group>
@get:ManyToMany(mappedBy = "id")
@get:JunctionTable
val friends: MutableSet<Person>
@get:Lazy
var about: String?
@get:Column(unique = true)
var uuid: UUID
var homepage: URL
var picture: String
}
43. EntityDataStore<Object>
• findByKey
• select / insert / update / upsert / delete
• where / eq, lte, lt, gt, gte, like, in, not …
• groupBy / having / limit / offset
• support SQL Functions
• count, sum, avg, upper, lower …
• raw query
44. @Test
fun `insert user`() {
val user = RandomData.randomUser()
withDb(Models.DEFAULT) {
insert(user)
assertThat(user.id).isGreaterThan(0)
val loaded = select(User::class) where (User::id eq user.id) limit 10
assertThat(loaded.get().first()).isEqualTo(user)
}
}
val result = select(Location::class)
.join(User::class).on(User::location eq Location::id)
.where(User::id eq user.id)
.orderBy(Location::city.desc())
.get()
val result = raw(User::class, "SELECT * FROM Users")
val rowCount = update(UserEntity::class)
.set(UserEntity.ABOUT, "nothing")
.set(UserEntity.AGE, 50)
.where(UserEntity.AGE eq 100)
.get()
.value()
val count = insert(PersonEntity::class, PersonEntity.NAME, PersonEntity.DESCRIPTION)
.query(select(GroupEntity.NAME, GroupEntity.DESCRIPTION))
.get()
.first()
.count()
45. CoroutineEntityDataStore
val store = CoroutineEntityStore(this)
runBlocking {
val users = store.insert(RandomData.randomUsers(10))
users.await().forEach { user ->
assertThat(user.id).isGreaterThan(0)
}
store
.count(UserEntity::class)
.get()
.toDeferred()
.await()
.let {
assertThat(it).isEqualTo(10)
}
}
with(coroutineTemplate) {
val user = randomUser()
// can replace with `withContext { }`
async { insert(user) }.await()
assertThat(user.id).isNotNull()
val group = RandomData.randomGroup()
group.members.add(user)
async { insert(group) }.await()
assertThat(user.groups).hasSize(1)
assertThat(group.members).hasSize(1)
}
46. spring-data-requery
• RequeryOperations
• Wrap EntityDataStore
• RequeryTransactionManager for TransactionManager
• Support Spring @Transactional
• Better performance than spring-data-jpa
• when exists, paging, not load all entities
47. spring-data-requery
• Repository built in SQL
• ByPropertyName Auto generation methods
• @Query for Native SQL Query
• Query By Example
• Not Supported
• Association Path (not specified join method)
• Named parameter in @Query (just use `?`)
(support next version)
48. Setup spring-data-requery
@Configuration
@EnableTransactionManagement
public class RequeryTestConfiguration extends AbstractRequeryConfiguration {
@Override
@Bean
public EntityModel getEntityModel() {
return Models.DEFAULT;
}
@Override
public TableCreationMode getTableCreationMode() {
return TableCreationMode.CREATE_NOT_EXISTS;
}
@Bean
public DataSource dataSource() {
return new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2)
.build();
}
}
49. Provided Beans
@Bean
public io.requery.sql.Configuration requeryConfiguration() {
return new ConfigurationBuilder(dataSource, getEntityModel())
// .useDefaultLogging()
.setEntityCache(new EmptyEntityCache())
.setStatementCacheSize(1024)
.setBatchUpdateSize(100)
.addStatementListener(new LogbackListener())
.build();
}
@Bean
public EntityDataStore<Object> entityDataStore() {
log.info("Create EntityDataStore instance.");
return new EntityDataStore<>(requeryConfiguration());
}
@Bean
public RequeryOperations requeryOperations() {
log.info("Create RequeryTemplate instance.");
return new RequeryTemplate(entityDataStore(), requeryMappingContext());
}
EntityCache 설정Tip :
개발 시에는 EmptyEntityCache,
운영 시에는 Cache2kEntityCache 사용
50. Use @Query in Repository
interface DeclaredQueryRepository extends RequeryRepository<BasicUser, Long> {
@Query("select * from basic_user u where u.email = ?")
BasicUser findByAnnotatedQuery(String email);
@Query("select * from basic_user u where u.email like ?")
List<BasicUser> findAllByEmailMatches(String email);
@Query("select * from basic_user u limit ?")
List<BasicUser> findWithLimits(int limit);
@Query("select * from basic_user u where u.name=? and u.email=? limit 1")
BasicUser findAllBy(String name, String email);
@Query("select u.id, u.name from basic_user u where u.email=?")
List<Tuple> findAllIds(String email);
@Query("select * from basic_user u where u.birthday = ?")
List<BasicUser> findByBirthday(LocalDate birthday);
}
51. Query By Example
BasicUser user = RandomData.randomUser();
user.setName("example");
requeryTemplate.insert(user);
BasicUser exampleUser = new BasicUser();
exampleUser.setName("EXA");
ExampleMatcher matcher = matching()
.withMatcher("name", startsWith().ignoreCase())
.withIgnoreNullValues();
Example<BasicUser> example = Example.of(exampleUser, matcher);
Return<? extends Result<BasicUser>> query = buildQueryByExample(example);
BasicUser foundUser = query.get().firstOrNull();
assertThat(foundUser).isNotNull().isEqualTo(user);
52. Query by Property
List<User> findByFirstnameOrLastname(String firstname, String lastname);
List<User> findByLastnameLikeOrderByFirstnameDesc(String lastname);
List<User> findByLastnameNotLike(String lastname);
List<User> findByLastnameNot(String lastname);
List<User> findByManagerLastname(String name);
List<User> findByColleaguesLastname(String lastname);
List<User> findByLastnameNotNull();
@Query("select u.lastname from SD_User u group by u.lastname")
Page<String> findByLastnameGrouped(Pageable pageable);
long countByLastname(String lastname);
int countUsersByFirstname(String firstname);
boolean existsByLastname(String lastname);
Note: Association Path is not supported
Note: Association Path is not supported
53. Exists
static class ExistsExecution extends JpaQueryExecution {
@Override
protected Object doExecute(AbstractJpaQuery query, Object[] values) {
return !query.createQuery(values).getResultList().isEmpty();
}
}
static class ExistsExecution extends RequeryQueryExecution {
@Override
protected @Nullable Object doExecute(AbstractRequeryQuery query, Object[] values) {
Result<?> result = (Result<?>) query.createQueryElement(values).limit(1).get();
return result.firstOrNull() != null;
}
}
Spring Data JPA - ExistsExecution
Spring Data Requery - ExistsExecution
54. Delete in JPA
static class DeleteExecution extends JpaQueryExecution {
private final EntityManager em;
public DeleteExecution(EntityManager em) {
this.em = em;
}
@Override
protected Object doExecute(AbstractJpaQuery jpaQuery, Object[] values) {
Query query = jpaQuery.createQuery(values);
List<?> resultList = query.getResultList();
for (Object o : resultList) {
em.remove(o);
}
return jpaQuery.getQueryMethod().isCollectionQuery() ? resultList : resultList.size();
}
}
Spring Data JPA - DeleteExecution
Load all entities to delete for
cascading delete
55. Delete in requery
static class DeleteExecution extends RequeryQueryExecution {
private final RequeryOperations operations;
DeleteExecution(@NotNull RequeryOperations operations) {
Assert.notNull(operations, "operations must not be null!");
this.operations = operations;
}
@SuppressWarnings("unchecked")
@Override
protected @Nullable Object doExecute(AbstractRequeryQuery query, Object[] values) {
QueryElement<?> whereClause = query.createQueryElement(values);
QueryElement<?> deleteQuery = applyWhereClause((QueryElement<?>) operations.delete(query.getDomainClass()),
whereClause.getWhereElements());
Scalar<Integer> result = ((QueryElement<? extends Scalar<Integer>>) deleteQuery).get();
return result.value();
}
}
Spring Data Requery - DeleteExecution
Use delete statements directly
cascading depends on DB
56. Future works
• Support Named parameter (ready)
• Support `@Param` in spring data
• Support complecated aggregation operations
• Requery for Apache Phoenix (done)
• CoroutineEntityStore (testing)
57. Resources
• requery.io
• spring-data-requery in github
• 2.0.x for spring-data-commons 2.0.x
• 2.1.x for spring-data-commons 2.1.x