Projects
From the meeting on Feb, 18th 2025, here are some of the software stacks that we are interested in contributing to.
Software Project | Description | Interests |
|---|---|---|
SLEEF
| Used by PyTorch and other projects for complex math operations (quad, dft, fft) on CPU. It supports RISC-V fully since 3.6.1 | Rivos - @Ludovic Henry |
OpenBLAS
| Industry standard for BLAS operations. Used by every AI/ML framework out there. It supports RISC-V RVV 128/256/512 bits already. The implementation is currently optimized for large square matrices which aren’t the typical shapes in AI/ML workloads. There is already support to pick different kernels based on the matrix shape for x86, we need to leverage that mechanism on RISC-V as well | Microchip - @Ken Unger ZTE - @Yunxiang Jia |
Eigen
| Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms. Used by PyTorch, Tensorflow, and Tensorflow-Lite (LiteRT). It doesn’t support RISC-V at all at the moment. There is an open issue and a corresponding open MR but there hasn’t been much activity over the past 5 months | Microchip - @Ken Unger |
oneDNN | It provides a DNN API (similar to cuDNN) and it’s used in lots of places. It is already functional on RISC-V (per the documentation), but it is still experimental and it’s unclear how optimized it is with RVV. | Rivos - @Ludovic Henry ISCAS - @fei zhang ZTE - @Yunxiang Jia |
XNNPACK
| There has been contributions from SiFive and Microchip, but contributions are slow to get merged. Contributions are being tracked in https://docs.google.com/spreadsheets/d/1PZAzBSqpdwoNgkxgrnmf5DDsZBzC_GEh/view We need to identify which intrinsics are critical to the models we care for. Notes:
| Microchip - @Ken Unger |
PyTorch CPU
| This work focused on PyTorch operators themselves, and not the dependencies used by PyTorch like OpenAI Triton, SLEEF, etc. A blocker is (was?) the availability of hardware to test on. Patches from SiFive were rejected for this reason There is an open PR to integrate cross-compilation support to CI. There would still be work to optimize many of the PyTorch Operators using RVV | Alibaba - @Binhua Wang ISCAS - @fei zhang ZTE - @Yunxiang Jia |
IREE | Interest is in the context of using it as a PyTorch compiler backend (see https://pytorch.org/docs/stable/torch.compiler.html for details) | Andes - @Ruinland Tsai |
OpenAI Triton |
| BOSC - @David Gao
|
Scikit-Learn | scikit-learn already works out-of-the-box on RISC-V. However to reach better performance, Intel provides scikit-learn-intelex on x86 which is based on oneDAL. Rivos has already done the work to accelerate oneDAL on RISC-V, but we now need to provide a similar plugin to scikit-learn-intelex for RISC-V to bridge the worlds of scikit-learn and oneDAL. | Rivos - @Ludovic Henry |
LiteRT (Tensorflow-Lite) |
| SiFive - @Hong-Rong Hsu |
Executorch |
| Andes - @Alan Quey-Liang Kao(高魁良) |
Numpy
| NumPy is the fundamental package for scientific computing with Python. With the Highway library, NumPy can leverage RVV (RISC-V Vector Extension) acceleration on RISC-V architectures. | ISCAS - @wang yang |
Knowhere | Knowhere is the core component of the open-source vector database Milvus, specializing in high-performance vector search. By integrating multiple underlying libraries (e.g., FAISS, HNSW, Annoy, etc.), it provides a unified vector computing interface for upper-layer applications. | ISCAS - @Liuyudong |
MNN | MNN is a highly efficient and lightweight open-source deep learning framework, specializing in high-performance on-device inference and training. By integrating multiple hardware backends (e.g., CPU, GPU, NPU, etc.), it provides a unified interface for efficiently executing deep learning tasks. | ISCAS - @Liuyudong |
Milvus | Milvus is a high-performance vector database built for scale. It powers AI applications by efficiently organizing and searching vast amounts of unstructured data, such as text, images, and multi-modal information. | ISCAS - @Liuyudong |
Faiss | Faiss is a library for efficient similarity search and clustering of dense vectors, developed by Facebook AI Research. It enables fast nearest neighbor search in high-dimensional spaces, making it suitable for applications such as image retrieval, recommendation systems, and large-scale machine learning. | ISCAS - @Liuyudong |
vLLM | vLLM is an open-source library for high-speed inference and serving of large language models (LLMs). Originally developed at UC Berkeley, its standout feature is PagedAttention, an innovative memory management algorithm inspired by operating system paging. This technology efficiently organizes the model's Key-Value (KV) cache, drastically reducing memory fragmentation and waste, which allows for significantly higher throughput compared to standard inference engines . | ISCAS - @Liuyudong |