Roadmap: Support for efficient numeric vector and matrix computations.

[sgaseretto]

Sorry if this isn't the correct place to post this. I saw that the roadmap mentions the implementation of computations for numeric vectors and matrices. This seems great for storing images representations and other types of latent representations and embeddings! What types of operations between matrices and vectors are planned for this? Will this somehow use the cube extension from postgres or it will be builded from scratch?

[1st1]

What types of operations between matrices and vectors are planned for this?

We only have very tentative plans for this. The idea is to add a proper multidimensional numeric array type with some ops exposed in EdgeQL. We still have lot's of open questions:

• should we use apache arrow for the memory format
• we could use numpy
• we could use the cube extension (or other relevant Postgres extensions)

What types of operations between matrices and vectors are planned for this?

Most definitely we'll not be building everything from scratch.

I'm curious what is your use case? Have you used the cube extension yourself?

[sgaseretto]

Three use cases that I've been working on are image search, and single and group recommendations serving.

1. Image Search: you get the image latent representation or embedding (dense vector) you want to search and compare that representation with the ones you have in your database using some distance metric (euclidean distance or cosine similarity) and return the TopN nearest representations to the one you are querying
2. Querying Recommendations: same as Image Search, but you compare the user latent representation with the products latent representations and get the TopN nearest products to that user
3. Querying Group Recommendations: get the embeddings from certain users, create a group representation by adding the users embeddings, and then perform a search similar to the one stated in point 2.

For both cases you can buid a model that perform s Locality-Sensitive Hashing to get the candidates ID, so if you have 100k image representations, that LSH model reduces the candidates to a few hundreds representations IDs, but then you have to perform the similarity calculation to get the TopN between those IDs you LSH returned

The problem with the Postgres cube extension is that it only supports a 100 Dimensions and some distance calculations are not implemented, like cosine similarity and pearson correlation

[1st1]

Thanks! A couple more questions to guide our future research:

The problem with the Postgres cube extension is that it only supports a 100 Dimensions and some distance calculations are not implemented, like cosine similarity and pearson correlation

Would something like numpy/pandas typically work for your use cases? If not Postgres/cube, what other DBs or solutions would you use?

[sgaseretto]

Numpy would probably work, I think that the two main bottlenecks are going to be speed and RAM usage, because you would need to serialize and deserialize the vectors to manipulate them with numpy, and also you will need to load a bunch of embeddings into memory to calculate the distances between them per vector. Frankly I'm not an expert in Databases, so probably I'm naively assuming that doing this operations inside the DB would be more performant, in terms of RAM usage. In recommendation systems solving this problem is very important since you are probably going to be doing this types of calculations thousands of times per minute.

Imagine this situation. To serve one recommendation per user you need to load 999 candidate vectors of 512 floats each and compare the user vector with this. This means loading 1000 vectors * 512 floats * 8 bytes = 4 megabyte per recommendation. If you are serving 1k recommendations a second you will already need to load 4GB to RAM, 5k would mean 20GB and so on. Also serving recommendations or search results has to be fast, like it should take you no more than 70 ms to present your results.

I haven't found any good DB that performs well for this types of problems, although the task is fairly ubiquitous, since recommender systems are implemented in lots of sites.

[hjalmarlucius]

I'm also in need of this feature. My use case is that I want to use EdgeDB as a database for storing output from a digital twin simulation engine:

• I will per time step and per compute node receive a computed tensor (e.g. with 24 elements shaped 2x3x4) that should be stored in the database.
• A data explorer tool will query the database (which also contains the relationships between all compute nodes, e.g. if they belong to the same component) to create visualizations - mainly as timeseries.
• Main tensor-related operations are 1) append-write, 2) slice, 3) aggregate (e.g. sum).
• Currently I'm storing tensors as an array and then reshaping based on strides manually. The absence of nested arrays means I'm now always appending each step to the same array which I expect to be a problem.

[abcnow]

Any update @1st1 on the vector implementation and roadmap timeline?