There are parallels between storing JSON data in PostgreSQL and storing vectors that are produced from AI/ML systems. This lightning talk briefly covers the similarities in use-cases in storing JSON and vectors in PostgreSQL, shows some of the use-cases developers have for querying vectors in Postgres, and some roadmap items for improving PostgreSQL as a vector database.

1. © 2023, Amazon Web Services, Inc. or its affiliates.
© 2023, Amazon Web Services, Inc. or its affiliates.
Jonathan Katz
Principal Product Manager – Technical
Amazon RDS Open Source
Vectors are the new JSON
or "Going beyond the Page"

2. © 2023, Amazon Web Services, Inc. or its affiliates.
© 2023, Amazon Web Services, Inc. or its affiliates.
Vectors are the new JSON
2

3. © 2023, Amazon Web Services, Inc. or its affiliates.

4. © 2023, Amazon Web Services, Inc. or its affiliates.
Magnitude

5. © 2023, Amazon Web Services, Inc. or its affiliates.
Direction

6. © 2023, Amazon Web Services, Inc. or its affiliates.
Why use vectors?
• Math
• Physics
• Maps
• Artificial intelligence / machine learning
6

7. © 2023, Amazon Web Services, Inc. or its affiliates. 7

8. © 2023, Amazon Web Services, Inc. or its affiliates. 8
0.003049829158165096,0.18639789248165428,0.25620939279058064,0.8474032584483915,0.728909751352397,0.5150510395040762,0.4462516533698704,0.5236709561811317,0.23075271031726885,0.2372686347975339,0.9375801900227572,0.9064239595
240302,0.4651187831953436,0.26254337449177356,0.8550190813318466,0.9841195123557434,0.8949629761265854,0.3042150462800528,0.057239707391303796,0.10891993000094757,0.6202587844948866,0.7137745836868454,0.06048318180182832,0.72
75440976627969,0.9478888345354122,0.8075292635669378,0.006104610242815056,0.723315680514812,0.8404533531441913,0.25125447598935935,0.7878795080344219,0.2611767393761504,0.7249823457880282,0.5664013738334184,0.6664423050495287
,0.047870779146162334,0.6024105618756934,0.28159282587919066,0.2603034349390896,0.2542046099573376,0.3327296553204988,0.2429052644111387,0.06034160939414335,0.0704690858202035,0.07525846779974898,0.5113090488711158,0.17784307
958376644,0.008055114879436331,0.7253841358982029,0.5892576733958386,0.031036548480514625,0.8514886217774027,0.5359628134105705,0.23253434065398793,0.9932524820134176,0.290096698529549,0.061493269072208534,0.19796591975942945
,0.5745454363733558,0.10500231825551154,0.8749785303875726,0.16050691467961897,0.6607092358366629,0.6959470185805223,0.8304841585326201,0.9906126039820897,0.37756396267006664,0.24750520102363893,0.7688257200202528,0.229018785
4669945,0.7383023205335917,0.9526504517225298,0.6702366304370244,0.12340838272000454,0.837277567807817,0.13355887990899973,0.16756984658473328,0.020480357387445025,0.7102474499378957,0.47830720614514277,0.7584869382921582,0.3
3627673352437526,0.33973495270900855,0.8041895249569642,0.8477249425000863,0.1840268911963996,0.6614288138464417,0.0733066286605144,0.1540691524669242,0.027117438127323368,0.8556511309682406,0.8365708233650331,0.9148876865925
715,0.27809873307734634,0.9847176522094117,0.8341376068583024,0.4884153769148796,0.49402832531031393,0.9759849942839054,0.3024687238313639,0.3054492792959067,0.9631850034636251,0.6306196638541337,0.25281927918295466,0.9633999
981843218,0.8556476268707236,0.35436292433753636,0.9181200874584725,0.5330211011157004,0.3659282554036771,0.5191548141957298,0.9933155783239833,0.6985272342940974,0.13341211270217812,0.14939654682403614,0.6996626768527356,0.1
5274920318616836,0.7375454247353765,0.12547466888770487,0.020867445809852114,0.08862531355325842,0.8594089835042844,0.467169285835908,0.3259292275603194,0.6754817412472605,0.23752820683506926,0.9235739428656018,0.600996868196
102,0.9826246759026915,0.3596323121373821,0.9926429393046163,0.45436365848015825,0.9537925268703233,0.7878438072069436,0.32655451673684865,0.18200719773833995,0.17466165185684446,0.5548376371379398,0.968462058702201,0.4360127
9039461005,0.3831529322604226,0.463364825678358,0.9972517349699217,0.3281245329407163,0.9106631366236755,0.7198211244102204,0.4306104787062708,0.23199225425841163,0.6140834474634502,0.811014846466751,0.91963436283595,0.679995
246870515,0.8567318394809504,0.14683252560984172,0.14260090503150025,0.8465205833865603,0.8549052513061426,0.8231241188837579,0.41924114155588654,0.38507506362948263,0.25006575098701234,0.0699639742348559,0.9833550311307562,0
.2605607961817462,0.259751219897133,0.02974752594215957,0.4000184436063705,0.56251539587252,0.7588095331041487,0.5247394729399737,0.6386582169689774,0.6753040885419601,0.34278485901716493,0.32020485063128845,0.425275092637406
26,0.14547296876436278,0.9141495998126814,0.040762738089558326,0.32204932557293375,0.8561755675889025,0.83711470922659,0.6558611621265165,0.9120719749549835,0.8783390682648751,0.8461287182939046,0.19516388006501373,0.90823287
55599889,0.5057372708578178,0.9256808350182766,0.7941729566984499,0.6312156320217639,0.3372050641335811,0.45407002325999457,0.0907182103421924,0.300577230348992,0.8878090275893804,0.3114290718440529,0.5082835005874564,0.90912
65859289592,0.5889511705440604,0.0731643995090181,0.7648932788544478,0.07398910725642338,0.37532303126425504,0.16537776734281096,0.5569586346091491,0.41320471229252576,0.5076657196988315,0.16068925028316272,0.3847130576569562
,0.43543125218091916,0.20087330502095924,0.5937208155983953,0.8364121127564523,0.1689155375268001,0.826672536335348,0.5184907315083152,0.8372109994079331,0.3265766880792995,0.4516165540976509,0.8988998457554018,0.728577359563
8751,0.5046728999487016,0.7202643135137343,0.09277559449311212,0.9863765116823693,0.05656950793722615,0.26912621193846675,0.574617488821012,0.6496403434996161,0.9496949766656257,0.08131020257688348,0.447922842165589,0.4348822
0063391125,0.1975887454376135,0.28999679910503673,0.6710527089809872,0.1981560761511254,0.7203622407001546,0.6892261426735438,0.5979394664255508,0.10251205002051123,0.601888419029919,0.5944410266337989,0.8940782482579479,0.10
38290594697493,0.32220747530470817,0.14736676773501856,0.9973485966907916,0.3130935141094717,0.30621340760343685,0.8178173879797406,0.5610692958400634,0.386045815840081,0.967519534919262,0.8097180412961045,0.14271245221528517
,0.3678693154775097,0.7779673784677215,0.6239712961871895,0.3259315559082765,0.7455054482396442,0.962644104725257,0.9138400820508927,0.4166565532574751,0.775627551221465,0.4272355508432639,0.43946813542828167,0.40726595029018
85,0.23280199025453285,0.9544162732488246,0.7903299555340588,0.518175708676786,0.40854152359527873,0.559741640565548,0.20424190854972935,0.9061262547687292,0.6645701182166341,0.9417612949664544,0.3286445198182655,0.0661663600
3648324,0.8575535235888658,0.9780448942056097,0.906878474768515,0.6836150303737369,0.09081361022805723,0.2563514823662274,0.8449437256183039,0.7375439247910052,0.17226960846450723,0.251759683310226,0.8416976841571362,0.183602
27348457414,0.8562937493383949,0.2953087294177408,0.11968508369259823,0.6069678436358927,0.9877152267663867,0.27230669196485735,0.8499940280590259,0.5321402710719028,0.4443626806404417,0.6492937309038496,0.6510203249326985,0.
19211109959844208,0.763934356818897,0.08368129338246888,0.28924073707477405,0.22268720990674495,0.3433845445499948,0.4109245245673492,0.1046155910313864,0.07657503894720463,0.49429763722934084,0.13795127372135596,0.1117575707
5977444,0.7050411510649539,0.6338950210353609,0.8713185455257246,0.5313368061200201,0.8819643864984599,0.7745177804106511,0.0626726238350983,0.22828785567392273,0.23592504355521982,0.8589077320621961,0.516653944907091,0.17131
219226574146,0.7913014103977858,0.5667533726397558,0.7471386821577681,0.28212564092399006,0.420452513398736,0.010283762019302145,0.22199748716045065,0.5662042042191722,0.7878722539548519,0.3418149766672123,0.05535447442751717
5,0.06587915283929746,0.9762432269886503,0.14083836839100528,0.8062479029933911,0.2610934713032229,0.26676251460956024,0.5373702653937364,0.7558546584516677,0.07565204292131611,0.3857456900574263,0.19486676744174147,0.6598951
373903503,0.5931976468336408,0.7007532126076583,0.431177727629251,0.32196004461643213,0.11636923570781832,0.26736440608024026,0.13902869217847424,0.9863974946645726,0.9367183779619204,0.5979123544639506,0.5957612994500785,0.1
007942253443872,0.4471694956922043,0.49227738044253755,0.3710392323565692,0.3083302449968812,0.30142992133120217,0.07513135879123212,0.0745710871699039,0.9753584092028262,0.6872321449386298,0.3013690191998748,0.68537127879475
08,0.2810608194143924,0.36719324243578555,0.9900196745730696,0.2995581604401316,0.05118911786611591,0.5901006755335594,0.9354925337581719,0.7831827067647659,0.5295572826622958,0.877399749074879,0.7229849844585559,0.5562242064
95262,0.08759996663518166,0.8385113135143705,0.5823150253024991,0.42793924572137954,0.17803833592182627,0.8114191217912143,0.3819904481571612,0.4945896490589128,0.3632353117172471,0.21192990341563345,0.7642874853025248,0.8687
117951383279,0.9775246638275519,0.9094460166851484,0.41439450351180795,0.7831566479261092,0.4166401099689523,0.8354457950736922,0.6448210109227475,0.07988475871585266,0.4529530027371038,0.9327107284948788,0.3534428053282923,0
.5078262077374696,0.6347768993059759,0.7313667268301707,0.11353959666563895,0.6990197817254433,0.28936586970647227,0.5078392143333978,0.7013959445391791,0.32886143696090997,0.9758973704822864,0.56380401177503,0.92651892607582
2,0.2838770887574107,0.25579145412399384,0.5262181909728092,0.756940652780294,0.7263718105964934,0.20790256344214697,0.29173264616980177,0.6237096498334083,0.6551044316369179,0.32577973645615543,0.8475413310666333,0.531446259
9579386,0.8884261026200484,0.9193022839479994,0.4274562716916215,0.4232826990646217,0.6428614697740933,0.29516246105833943,0.49128289620072607,0.766792234884953,0.9257844666811827,0.217110898687352,0.7351022356386885,0.801423
2451978316,0.5136552177138505,0.1204561525036354,0.5906654571634355,0.4669067015637509,0.13337295607131594,0.4644932295617963,0.49771730573777617,0.006138980193121313,0.7179200071510401,0.4054289273828431,0.13075866432562222,
0.285819886194453,0.9610065864773567,0.6307008303612776,0.9317299086944182,0.32493745347351677,0.36938322317655903,0.9487034160287919,0.8956663435211674,0.576950512486242,0.2044998575366499,0.8268154915267445,0.93399007412639
53,0.5287662643860003,0.04401588702004133,0.031869671118101905,0.10932950531859476,0.9015072253500662,0.6831864617088179,0.3830433812043026,0.10929357802114836,0.5483989715204345,0.06937497005475635,0.17543350260351076,0.9702
931680744307,0.31830118555081555,0.33100272148012166,0.3867797932939965,0.9452420923577414,0.10448864271452152,0.4644021363303388,0.41902724092989274,0.6528552076589271,0.8329300403970947,0.19567210841596605,0.945391035696346
2,0.07648646003400117,0.7088000320729222,0.08525142339897585,0.7926614929870937,0.37533575924477347,0.9712075380467198,0.314161010077175,0.5636365544597624,0.8118455267394573,0.18167043282866047,0.33447618370067644,0.53739801
77527073,0.8207760090494212,0.5598771155060618,0.812458723315661,0.9328809520319652,0.48548827391470795,0.589538317374938,0.9989134212732473,0.8066754110779186,0.5361955264216469,0.44706185290136347,0.8638674649651144,0.11772
212031853257,0.7373123094378364,0.2964179382954839,0.3958614710526014,0.033620851569537535,0.9336075991629791,0.07556724910015333,0.3323710034423364,0.5953882717517018,0.5278470851785038,0.6444871267215646,0.02765478052902281
5,0.8850327220213501,0.16931234747461232,0.5340673362608825,0.02628332727484306,0.8542332140725506,0.5494555230603417,0.6314945868437789,0.3714973660250145,0.7393204833479103,0.48755770442043556,0.9543687133889094,0.492261579
0703624,0.28988577702968144,0.7083030437722719,0.8364820479067632,0.26442432836035223,0.923539740473899,0.5804786632967982,0.8973611414769564,0.19090029594385172,0.9510858616975355,0.5205083047693755,0.7603556975149317,0.5825
186251699712,0.5237620021321732,0.1382955047996397,0.6759274053081654,0.7125068099176595,0.28196614561746003,0.5910852667039386,0.6941303825773772,0.5590181096403093,0.44337019902363295,0.0420720293796073,0.39999920949987455,
0.4152904743104564,0.3835150394646547,0.8257315199272988,0.20520912672005664,0.13770565792973244,0.525472587292029,0.5859051326757871,0.8971099388662438,0.49808119463056144,0.47418305344412204,0.6632131152031171,0.34084957549
33867,0.2159830200386672,0.9784792267668365,0.7062373744186736,0.5612091756778526,0.055825554989286985,0.16807129411874655,0.41004145527270097,0.688008043203375,0.4962868413950652,0.04798678592732131,0.44305450369300203,0.613
1679288963952,0.1084916435499288,0.9088673748726279,0.7093331164130028,0.5000408976944115,0.9350131149208245,0.8634689880409212,0.7610502159510695,0.2183175929625989,0.9426464930280254,0.398670585712793,0.2821887816279194,0.2
0328208191184416,0.4548741358447934,0.6548715754290697,0.4184107296671691,0.3995297162154898,0.35985286475851197,0.14307473061241183,0.3426227796135173,0.7303046335724659,0.6696198440279808,0.0820622313135857,0.78669208302376
25,0.7087577215211311,0.5871649173453939,0.29735360309864234,0.5060200361378087,0.291713126771036,0.8593132510665988,0.2491874611967546,0.3973333635200156,0.17562578418895924,0.6723851134095646,0.8433206332882293,0.5862925569
757031,0.39460302185096197,0.3298574619385626,0.08633999822822602,0.5186693958933226,0.13856413475384244,0.16037753045039338,0.7524108020739817,0.4083528740397604,0.14336197236726633,0.5922296134590717,0.17254185951185264,0.2
5187672624444346,0.18215411670198733,0.726615554336707,0.19775948066525473,0.6502171988473684,0.11727720046996382,0.504884864310899,0.7468997428055957,0.47510705275687926,0.5537603917188179,0.23027474644279167,0.0665294161855
5985,0.7729963399649726,0.4094521145359735,0.6370513839150114,0.6088910155676217,0.4621325659011113,0.11319174753695194,0.341564487945039,0.9934636465304294,0.297227753569981,0.7328244053776913,0.6311450999071475,0.8440901108
017158,0.15514797351432819,0.08104602596240085,0.49662481729486174,0.8876186901886598,0.04044740877096942,0.8516169198563475,0.26929077648597755,0.8282467066551362,0.8809442986462663,0.5945587511571162,0.4381976893835038,0.45
645891923005877,0.4435954618268134,0.9803416732282741,0.28194380839847,0.7604786292220354,0.7654898505617886,0.3541891007560096,0.011414697414370067,0.5449695861628214,0.7827950459053703,0.2075164240522085,0.6758649278280906,
0.0554439805055722,0.22402408179082656,0.44984092489657,0.20142611099850072,0.2021725378636603,0.8882387955223372,0.4475422979857555,0.5042078339514582,0.2847085559918412,0.6820903566163103,0.7925305807392036,0.62322889648881
49,0.2634052122838142,0.9733630373009099,0.9017300092991363,0.5532375674290826,0.9991261787887424,0.11750396889071268,0.44547741255642137,0.8039341980687489,0.8939500761340611,0.3151848911928674,0.5182512914343462,0.378205264
5958096,0.6869011397829183,0.10407205957183052,0.5480067699689997,0.6338475706109143,0.9780782242518598,0.8206228388017074,0.4808312104378487,0.485162738616598,0.32610349576788167,0.7849523968684302,0.4563195194072378,0.30511
80678214003,0.4572974238537455,0.9638166279484572,0.32728609759492855,0.22600814737325337,0.47575953999646003,0.606858294386381,0.18709606637543885,0.9053451016853451,0.7225714493761046,0.1859076951207861,0.9015509000938309,0
.15081182034774443,0.6162413221106817,0.7058872551659015,0.2430669685977378,0.5884148017294031,0.9475041161074529,0.6114365999460425,0.9766288110478101,0.14447912926772588,0.497530656902228,0.551571242000545,0.424096282623402
5,0.9059704151167303,0.8129579833153429,0.8544592783288429,0.050935102703213886,0.6640650997798936,0.5830346238565767,0.8278167869285475,0.28298784896973217,0.450304101090115,0.41830426007102517,0.7626792333938361,0.516796149
1617277,0.7885632739167967,0.2869811219339624,0.5577234652871645,0.4721387833806787,0.43309028577887787,0.3004770094749887,0.17933809587426808,0.12740368744490382,0.5947491456227318,0.595024606540278,0.1638395722206205,0.3453
285042793155,0.20426355588516643,0.38923130948650453,0.19030229808697285,0.44731342822910136,0.5806663041043443,0.8305436981410423,0.49623415203229726

9. © 2023, Amazon Web Services, Inc. or its affiliates.
Dimensionality
• Machine learning algorithms create vector embeddings of "many
dimensionalities"
§ 256
§ 384
§ 768
§ 1536
§ 2048
§ 60K+
• Dimensionality keeps information about a particular attribute
9

10. © 2023, Amazon Web Services, Inc. or its affiliates.
Searching vector embeddings
• Vector normalization
§ Magnitude of 1
• Distance function
§ Euclidean distance (L2)
§ Cosine distance (cosine similarity)
§ Inner product
§ Taxicab / Manhattan (L1)
§ Chebyshev (L-inf)
10

11. © 2023, Amazon Web Services, Inc. or its affiliates.
K-nearest neighbor
SELECT *
FROM table
ORDER BY $VECTOR <-> embedding -- distance
LIMIT 5; -- k
11

12. © 2023, Amazon Web Services, Inc. or its affiliates. 12
100,000

13. © 2023, Amazon Web Services, Inc. or its affiliates. 13
1,000,000

14. © 2023, Amazon Web Services, Inc. or its affiliates. 14
100,000,000

15. © 2023, Amazon Web Services, Inc. or its affiliates. 15
1,000,000,000

16. © 2023, Amazon Web Services, Inc. or its affiliates.
Vector indexing methods
• ~20 years of modern research searching high dimensionality vectors
§ FAISS – open-source library of vector search algorithms
§ Leverage CPU/GPU acceleration
• "Approximate nearest neighbor" (ANN)
§ Tradeoff on performance / recall
• IVF FLAT (inverted indexes)
• HNSW (Hierarchical inverted small worlds)
16

17. © 2023, Amazon Web Services, Inc. or its affiliates.
pgvector
• Open source extension that provides vector data type, distance
operations, and indexing
• IVF FLAT index
• Can adjust performance / recall tradeoff through "probes"
• Connectors to interface with many programming languages
17

18. © 2023, Amazon Web Services, Inc. or its affiliates.
Quick recap on vectors in machine learning
• Vectors encode information about objects (text, images, videos)
• Vectors can have high dimensionality
• "Similarity search" is a popular operation
• Data sets can become very large
• Indexing techniques that can accelerate search at cost of "recall"
18

19. © 2023, Amazon Web Services, Inc. or its affiliates.
© 2023, Amazon Web Services, Inc. or its affiliates.
Going beyond the page
19

20. © 2023, Amazon Web Services, Inc. or its affiliates.
How big is a PostgreSQL page?
20
8192*
8192 – sizeof(page header)

21. © 2023, Amazon Web Services, Inc. or its affiliates.
How big is a 1536 dimensional vector?
21
1536 * 4 + 8 = 6152

22. © 2023, Amazon Web Services, Inc. or its affiliates.
How big is a 2048 dimensional vector?
22
2048 * 4 + 8 = 8200
X

23. © 2023, Amazon Web Services, Inc. or its affiliates. 23
But we can TOAST!

24. © 2023, Amazon Web Services, Inc. or its affiliates. 24
But we can TOAST (an index)!
X

25. © 2023, Amazon Web Services, Inc. or its affiliates.
What does this mean?
• PostgreSQL can only index vectors up to a certain dimensionality
without reduction
• Query plan costing becomes something developers need to worry
about
§ Cost of a sequential scan on a toasted column vs. index page
25

26. © 2023, Amazon Web Services, Inc. or its affiliates.
What can we do?
• Native support for vector operations in PostgreSQL – Functions / data types
• Mechanisms for managing vector data that goes beyond the page
• Performance: Indexing methods. Query plan costing. Parallelism (builds and
queries)
• Build on CPU acceleration work.
§ GPU acceleration – extension?
• Contribute to extensions (e.g. pgvector) / vice versa
26

27. © 2023, Amazon Web Services, Inc. or its affiliates.
© 2023, Amazon Web Services, Inc. or its affiliates.
Thank you!
27