Top-K off-policy correction for a REINFORCE recommender system

1. 73 *32 0 3 9 0 7
-
- 9 7 3 3713 e k S
df
a c
TN M [ i . / ] C

2. - . - -

3. 0 C - 2 . CD 2C
• -
• 2 9
• 10 -
• gd
• a lp W L GSGN C 2C 2 C fmb
• nk R G c eoNiM hN M3 2 I
• C , 2 C2 02

4. . 3 34 1-2 1 1 3 .
&
• kMN k
31 e
i
a k
k
S C d

5. 4 - 45 2.3 21 2 4-
&
• e t S t
42- C N o
rd C
kM t
t i
i t
C as np

6. - - - .- -
&
6
6

7. 0 - 50 . 0 10 -0
&
C
7 7 A
A
A

8. 0 - 50 . 0 10 -0
&
M
C
AN
8 8
MN
8 8

9. 0 - 50 . 0 10 -0
&
N
N
M
M
AC
9 9
9 9
N
N
S
S

10. 0 0 , 0 ,
l -
Ø . dw NO T c o
Ø 0 -, 100 -,. dw iO pM- , C
Ø YkN- , s cf 0 , 0 l
l
Ø -0 NemtyL bM gNS ty u r
Ø N M n - , sN a l

11. C . CE A B ) A A C
• D ( B A 2 C
R V Ma
S M d
• 1 F ( B A 3 - , 3) ) C
e Ma
a I S OQP NMc

12. O / 8 0 L GA + GF G8
l 1 B G 2 CC 8 N C 6- B 7
Ø t e]d b [ac gurov hw g f eik hy
l 2 CC 8 8 G 8 C 6 8 B 7
Ø xn l RTS/0 2 S/ 8 U. mp
l 1 F B N 8B 8 G B8 G CC 8 6 L8C 8 8 7
Ø + C 8 G 8B M 8B 8 F B N 8B 8 G hy
F G C8 G s

13. - . - -

14. 4 . - . 1 . . .-
C
:
C:M
S N

15. . - . 1 . . .-
C M
5 a
5
5 S N S :

16. ( . , ) 6 6 6,
M MSN
• 6,1 - 6: C M

17. ( . , ) 7 ,
• MS CN
• ,1 - : MS T

18. - . - -

19. - 1 1 - - .- 1 -
- ( )
• C 9

20. - 2- 0 - .- -
- ( )
• M C
M N C

21. - 1 12- - .- 1 -
-
• M C
M
( ) -
N C

22. . .- * . 2 *
• *1., . * , C N
• T ie M C d M-. * ca
b
h S N

23. .1 3 1
-
• * 2 - , 1 1- M S C
• j N a e MN. ,1 db
c h
a i T
-

24. 2. .- . 1 4 2
• M C F
.

25. - 5 )5 1 5 1
• x , 5 ] 25 1 N y bnl
. 2 1 5 [ow d p Ob US
• M us[ NmMU bP ] v,
• r ] ( p cS hfg dt
] 1Fkei d p
( C 5 a

26. - (-,. )- 6 2 .
• o C MN S s n s k
• k a
d
d it
e r C p

27. . , 7 ,
- ) )( )
• K C
• 2 -

28. , ( . ) 8 .
- ) )( )
• - 8 KPN 22 M C
O

29. 9 2 - 2 . 2 2 -2
• kac l S mdh
• kac S l mdh i N
M 22 , 2 , - 2 - CN e

30. . 3:
- I N -
- W PBC
S M - 0

31. . 3:
1 - I N -
1 - W PBC
S M - 1

32. - . - -

33. , 3 32 3 = = =
1 )( -- (
• k r vxtS sf xt
• k sf 23C eN
• u SK abcd
• F 1 1 ==31 oSp
1 1 ==31 ] [ wgl nyM
3 = 1 23C i
h O
- . =3

34. , 1 1 3 1 4 3
( - )
• l g eN d[ e
• l
• mN M
• = 3 31 Nak
= 3 31 C FKN 3 S i]
- . 32= 1

35. C - . C C
( )
• eu 1 5 ak [ ns]
• (o akY
• = C dY,) (
Sl M ] c Ymt
,) ( S p KM S
C 6F 6 6C ,) NibTgr
2 3

36. - . - -

37. 7 3- - 3 7
• Y lik M i
• - 3 3 T - j NMen v
d 3- u K e B 3 3 7 rp
• rp B 3 3 7 cb w
• a S, . M h Mty os C

38. - . - -

39. d AV MZ ML I MZ]V R 2V WV IZ V
(.
F G T 2PM 0SM 1M[ZMS I[S 2V OZV AIOI 9I 5 I KV 1MSSMZZ 4L 7 2P .
BVW : g VS K 2V MKZ V NV I 48 5 24 MKVTTM LM A ZMT
8 BPM B]MSNZP 02 8 ZM IZ V IS 2V NM M KM V DMJ AMI KP I L 3IZI O DA3 a .
5MJ [I . MSJV[ M C82 0[ Z IS I
F G BVJ I AKP IJMS 0L ZP A]IT IZPI 0 P[LMMW A OP I 2PI LIR I L BPV ZM 9VIKP T
MKVTTM LIZ V 0 B MIZTM Z / 3MJ I O MI O I L 4 IS[IZ V
8 VKMML O VN ZPM (( L 8 ZM IZ V IS 2V NM M KM V 8 ZM IZ V IS 2V NM M KM V IKP M MI O
CVS[TM )- 82 a , b ,.
F(G 0L ZP A]IT IZPI 0R PI : P IT[ ZP 0SMRP 0OI ]IS V 3[L R 9VP I ONV L
3IT M 9V M I L 8TML E ZV[ ,
g WVS K M IS[IZ V NV SIZM MKVTTM LIZ V
8 0L I KM M[ IS 8 NV TIZ V VKM O A ZMT
F)G 0SM I L M 6 SVZZM 2SfTM Z 2ISI[ e M BPVTI MLMSMK 0SM I L M 0J IPIT I L A TV 3VSSf -
h M 0 1 ZM Z O NV MKVTTM LM A ZMT
8 VKMML O VN ZPM 4SM M ZP 02 8 ZM IZ V IS 2V NM M KM V DMJ AMI KP I L 3IZI O 02 .-b

40. 8 4 -4 52 .4 8 8 98 2 58
0 1 a (
ek[ i I o ]
nN d MA CS ( )