This document contains slides from a PLDI 2017 tutorial on vectorization with LMS using SIMD intrinsics. It discusses Single Instruction Multiple Data (SIMD) parallelism and how to implement SIMD intrinsics in LMS. Some challenges include handling large classes, automatically producing read/write effects, type mappings for unsigned values and pointers, and implementing arrays instead of general containers for the DSL. The slides provide examples of scalar and SIMD-vectorized code for addition, as well as an overview of SIMD instruction sets like AVX and the large number of intrinsics that need to be supported. It proposes generating intrinsics automatically from metadata to address porting them all to LMS.

1. PLDI 2017 Tutorial Session
Vectorization with LMS:
SIMD Intrinsics
Alen StojanovDepartment of Computer Science,
ETH Zurich, Switzerland

2. 2
SISD
SIMD
1
3
2
4
1
3
2
4
1
3
2
4
1
3
2
4
1
3
2
4
1
3
2
4
What is SIMD?
Single Instruction
Multiple Data

3. 3
SISD
SIMD
1
3
2
4
1
3
2
4
1
3
2
4
1
3
2
4
1
3
2
4
1
3
2
4
AVX x4
#define T double
void add(T* x, T* y, T* z, int N) {
for(int i = 0; i < N; ++i) {
T x1, y1, z1;
x1 = x[i];
y1 = y[i];
z1 = x1 + y1;
z[i] = z1;
}
}
Scalar
#define T double
void add(T* x, T* y, T* z, int N) {
for(int i = 0; i < N; i += 4) {
__m256d x1, y1, z1;
x1 = _mm256_loadu_pd(x + i);
y1 = _mm256_loadu_pd(y + i);
z1 = _mm256_add_pd(x1, y1);
_mm256_storeu_pd(z + i, z1);
}
}

4. 4
SISD
SIMDAVX x4
#define T double
void add(T* x, T* y, T* z, int N) {
for(int i = 0; i < N; ++i) {
T x1, y1, z1;
x1 = x[i];
y1 = y[i];
z1 = x1 + y1;
z[i] = z1;
}
}
Scalar
#define T double
void add(T* x, T* y, T* z, int N) {
for(int i = 0; i < N; i += 4) {
__m256d x1, y1, z1;
x1 = _mm256_loadu_pd(x + i);
y1 = _mm256_loadu_pd(y + i);
z1 = _mm256_add_pd(x1, y1);
_mm256_storeu_pd(z + i, z1);
}
}
LBB0_3:
movsd (%rdi,%rax,8), %xmm0
addsd (%rsi,%rax,8), %xmm0
movsd %xmm0, (%rdx,%rax,8)
incq %rax
cmpl %eax, %r9d
jne LBB0_3
LBB0_3:
vmovupd (%rdi,%r10,8), %ymm0
vaddpd (%rsi,%r10,8), %ymm0, %ymm0
vmovupd %ymm0, (%rax)
addq $4, %r10
addq $32, %rax
addq $1, %rcx
jne LBB0_3

5. • MMX
• SSE / SSE2 / SSE3 / SSSE3 / SSE4.1 / SSE4.2
• AVX / AVX2 / AVX-512
• FMA / KNC / SVML
8x float
4x double
32x 8-bits
16x 16-bits
8x 32-bits
4x 64-bits
256-bit
AVX
4x floats
2x doubles
16x 8-bits
8x 16-bits
4x 32-bits
2x 64-bits
SSE
operands
for each

6. 6
That’s not all
Shuffles:
• _mm256_permutevar_pd
• _mm256_shufflehi_epi16
• …
Strings:
• _mm_cmpestrm
• _mm_cmpistrm
• ..
Bitwise operators:
• _mm256_bslli_epi128
• _mm512_rol_epi32
• …
Statistics:
• _mm_avg_epu8
• _mm256_cdfnorm_pd
• …
Logical:
• _mm256_or_pd
• _mm256_andnot_pd
• …
Crypto:
• _mm_aesdec_si128
• _mm_sha1msg1_epu32
• …
Loads:
• _mm_i32gather_epi32
• _mm256_broadcast_ps
• …
Stores:
• _mm512_storenrngo_pd
• _mm_store_pd1.
• …
Casts:
• _mm256_castps_pd
• _mm256_cvtps_epi32
• …

7. 7
There are a lot of SIMD instructions
AVX-512 has 3519 intrinsics

8. How do you port all intrinsics into LMS?
Ivaylo Toskov
ETH Zurich
Idea #2: Generate them automatically
Idea #1: Get a Master student to do it

9. 9
data-3.3.16.xml

10. Challenge #1
Scala chokes on big classes ~ 64kB
limit for a method
• Split the implementation
into multiple classes
• Make one trait inherit all
split classes

11. Challenge #2
LMS has read / write effects
• Produce the effects
automatically using the
category data in the Intel
Intrinsics Guide
<intrinsic tech='AVX' rettype='__m256d' name='_mm256_loadu_pd'>
<type>Floating Point</type>
<CPUID>AVX</CPUID>
<category>Load</category>
<parameter varname='mem_addr' type='double const *’ />
<description>
Load 256-bits (composed of 4 packed
double-precision (64-bit) floating-point elements)
from memory into "dst". "mem_addr" does not need
to be aligned on any particular boundary.
</description>
<operation>
dst[255:0] := MEM[mem_addr+255:mem_addr]
dst[MAX:256] := 0
</operation>
<instruction name='vmovupd' form='ymm, m256’ />
<header>immintrin.h</header>
</intrinsic>

12. Challenge #3
Type Mappings – unsigned?
• Use Scala Unsigned for
unsigned operations.
Challenge #4
Pointers?
• Disallow and use memory
offsets instead
Challenge #5
Implement Arrays only?
• Abstract containers for the
need of the DSL
Challenge #6, #7, ...
Try to think of everything?
• Checked.

13. 13
https://github.com/ivtoskov/lms-intrinsics

14. How do we make use of
the intrinsics ?

15. 15
https://github.com/astojanov/lms-tutorial-pldi