Repository navigation
Conversation
…iniTensor#2 (Operators) - Tensor: implement load, isContiguous, view, permute, slice - Operators: implement argmax, embedding, linear, rms_norm, rope, self_attention, swiglu - All CPU implementations support F32, F16, BF16 data types - All tests pass
- Add Qwen2 model C++ implementation with KV Cache support - Add Python bindings for model inference - Add rearrange operator for KV cache management - Implement weight loading from safetensors files - Note: Full testing requires more system memory (model is ~3.5GB)
- Implement chunked reading for large tensors (4MB chunks) - Avoid data type conversion (keep bfloat16 as uint16) - Add proper memory cleanup with gc.collect() - Add C++ side weight loading function declaration - Model loading now works successfully
- Create CUDA Runtime API implementation - Add CUDA resource management (cuBLAS, cuDNN) - Implement CUDA kernels for all operators - Add NVIDIA compilation configuration - Create comprehensive CUDA documentation
Implement CUDA runtime APIs and operators for NVIDIA/MetaX. Fix cuBLAS F16/BF16 accumulation, causal attention mask offset, and argmax reduction for large vocab sizes. Add CUDA device link for the shared library.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
总结
本 PR 完成作业 #4：为 LLAISYS 集成 CUDA。
实现内容：
CI 状态
CI 在 Windows 和 Ubuntu 上均通过，Assignment #0-#3 全部通过。
最新运行：https://github.com/kadadsns/llaisys/actions/runs/31293416784
支持平台与状态
复现流程
详细复现步骤与结果见
ASSIGNMENT_REPORT.md和ASSIGNMENT_REPORT_ZH.md。主要改动
Makefile.metax）test/test_ops.py