DeepReinforce Open Sources Agentic Programming Series Model Ornith-1.0

This breakthrough research completed by the DeepReinforce team was published in December 2025, with research ID arXiv:2512.02551v1. The main authors of the paper include researchers Su Songqiao, Sun Xiaofei, Li Xiaoya, Wang Albert, Li Jiwei, and Shen Chris. This study is the first to prove that artificial intelligence can surpass NVIDIA's official optimization libraries in matrix calculations, the most core operation of GPUs, holding significant meaning for the entire AI and high-performance computing field.

Imagine you are cooking in a kitchen, needing to quickly chop vegetables, stir-fry, season, and perform various other operations. In the world of computers, the GPU (Graphics Processing Unit) is like a super kitchen, and matrix multiplication is the most important "main dish preparation technique" within it. Whether training AI models or running large language models like ChatGPT, almost all important calculations rely on this "main dish" of matrix multiplication.

For a long time, NVIDIA, as the absolute hegemon in the GPU field, has been like a top chef, carefully optimizing these core calculation codes. Their cuBLAS and cuBLASLt libraries are widely used by developers worldwide. These libraries have been meticulously tuned by countless engineers over many years, and their performance has reached an extremely high level. However, the DeepReinforce team proposed a bold idea: Can artificial intelligence learn to write these core codes itself, even surpassing the optimization level of human experts?

This idea sounds like teaching a robot to cook, not only as well as a human chef but even better. In the field of computer programming, this challenge is particularly difficult because GPU programming requires a deep understanding of hardware architecture, mastery of complex memory management, thread scheduling, cache optimization, and other technologies. Different matrix sizes require completely different optimization strategies, just like cooking different ingredients requires different heat and techniques.

The research team developed a system called CUDA-L2, which combines large language models and reinforcement learning technology. It can be understood as an AI chef that knows how to program; it not only understands basic "cooking skills" (programming knowledge) but can also discover better "recipes" (optimization strategies) through continuous trial and improvement. The system works like a tireless experimenter; it generates tens of thousands of different code solutions, each actually run and tested on the GPU, and then learns which methods are more effective based on performance.

To verify the system's capabilities, the research team designed a comprehensive testing environment. They selected 1000 different matrix configurations for testing, covering various situations from small to ultra-large matrices, including all matrix sizes actually used in current mainstream AI models such as Qwen, Llama, and DeepSeek. This is like testing a chef's true level in different kitchen environments with different ingredients and portions.

The test results were shocking. In "offline mode" (continuously executing calculation tasks), the code generated by CUDA-L2 was on average 22% faster than PyTorch's standard matrix multiplication function, 19.2% faster than NVIDIA's cuBLAS library, 16.8% faster than the more advanced cuBLASLt library, and even 11.4% faster than the auto-tuned version of cuBLASLt. In "server mode" (simulating intermittent calculations in actual applications), the performance improvements were even more significant, reaching 28.7%, 26.0%, 22.4%, and 15.9% respectively.

What do such performance improvements mean? If calculation speed is compared to a car's driving speed, then CUDA-L2 is like accelerating a car originally traveling at 100 km/h to 115-130 km/h. For AI training and inference tasks that require massive calculations, this improvement can significantly reduce calculation time and power consumption, lowering costs.

I. Unveiling the Mystery of Matrix Multiplication

To understand the importance of this research, we first need to understand the status of matrix multiplication in modern computing. Matrix multiplication is like concrete pouring in construction engineering; it is the foundation of almost all complex calculation structures. Whether it is image recognition, speech recognition, or natural language processing, there are massive matrix operations supporting the backend.

In the world of GPUs, there is a special type of matrix multiplication called HGEMM (Half-precision General Matrix Multiply). This calculation uses 16-bit numerical precision, like building a house with lighter materials, greatly improving calculation efficiency while maintaining sufficient precision. Modern AI models almost all rely on this calculation method to achieve fast training and inference.

The process of a GPU executing matrix multiplication is like a highly organized factory assembly line. First, huge matrices are divided into smaller "tiles," like cutting large pieces of fabric into small pieces for easier sewing. Each GPU thread block is responsible for calculating one output tile, which is further subdivided internally to match the dimension requirements of the GPU's tensor cores.

The entire calculation process is divided into three main stages. The first stage is data preparation, where the system loads matrix tiles from global memory into shared memory, like moving raw materials from a warehouse to a workbench. The second stage is the main calculation loop, where data flows between multi-level memories, from shared memory to registers, and then the tensor cores execute the actual multiply-accumulate operations. The third stage is result write-back, where the accumulated calculation results are written from registers back to shared memory, and finally transmitted to global memory.

This process seems simple, but it actually involves a large number of optimization details. Different matrix sizes require different tile sizes, different memory access patterns, and different thread scheduling strategies. Just like cooking, the same ingredients will produce completely different effects under different heat and time conditions.

II. The Courage to Challenge NVIDIA's Official Libraries

NVIDIA's cuBLAS library is like a classic old-brand restaurant, meticulously polished over decades, with every "dish" (algorithm) reaching an extremely high level. This library contains a large amount of hand-optimized core code, developed and maintained by world-class GPU programming experts. When deep learning frameworks like PyTorch execute matrix operations, they internally actually call the cuBLAS library.

cuBLAS provides multiple matrix layout options, mainly including NN (Normal-Normal) and TN (Transpose-Normal) layouts. This is like arranging tableware in different ways, each having its advantages in different situations. For each matrix configuration, cuBLAS automatically selects the optimal layout and algorithm.

The more advanced cuBLASLt library provides finer control capabilities. If cuBLAS is like a standard menu at a high-end restaurant, then cuBLASLt is like a private chef service that can be customized according to guest preferences. It exposes more algorithm options, allowing developers to perform deeper performance tuning.

cuBLASLt provides two usage modes. The first is heuristic mode, where the system recommends the best algorithm based on matrix configuration, like a senior sommelier recommending wine based on the dish. The second is auto-tuning mode, where the system tests up to 100 different algorithm candidates and selects the one with the best performance, like having 100 chefs make the same dish and then choosing the most delicious version.

The research team chose to challenge these time-tested libraries, requiring great courage and technical strength. This is like a young chef deciding to challenge the signature dish of a Michelin three-star restaurant; it not only requires deep technical skills but also innovative thinking.

III. How the CUDA-L2 System Works

The core idea of the CUDA-L2 system is to let artificial intelligence learn to write high-performance GPU code. This process is like cultivating a programming genius, starting from learning basic knowledge and gradually developing to creatively solve complex problems.

The first key component of the system is a code generator based on large language models. The research team chose the DeepSeek 67B model as the foundation, equivalent to selecting an "apprentice" with strong language understanding capabilities. Then, they performed continuous pre-training on this model, feeding it a large amount of CUDA programming code, including implementations from well-known libraries such as PyTorch, ATen, and CUTLASS, as well as various GPU programming examples collected from the internet.

This process is like letting a smart student read all relevant programming textbooks and excellent code examples. To help the model understand the meaning of these codes, the research team also used the Claude Sonnet model to generate code descriptions, pairing each segment of code with detailed explanations. Additionally, the system integrates retrieval-augmented functionality, able to query relevant documents and examples when generating code, like a programmer consulting a manual while programming.

Next is the reinforcement learning training phase, which is the most critical part of the entire system. Reinforcement learning is like training an athlete, improving performance through continuous practice and feedback. The system is divided into two stages of reinforcement learning training.

The first stage is general kernel optimization training. The system learns to optimize various types of GPU kernel code, including linear algebra operations, convolution operations, reduction operations, element-wise operations, attention mechanisms, and so on. This is like letting a chef learn various basic cooking skills, mastering foundational skills such as chopping, stir-frying, and seasoning. Each generated kernel code is run and tested on actual GPU hardware, and the system adjusts the generation strategy based on execution speed.

The second stage is specifically trained for HGEMM matrix multiplication. The system focuses on optimizing matrix multiplication of different configurations, discovering the best optimization strategies through extensive experiments. This is like specifically training a chef to make a signature dish, achieving perfection through repeated practice.

The system's reward mechanism is designed very cleverly. The main reward comes from the execution speed of the code; the faster the speed, the higher the reward. At the same time, the system also checks numerical accuracy to ensure the generated code is not only fast but also correct. Additionally, code length is also taken into consideration, encouraging the system to generate concise and efficient code.

IV. Optimization Techniques Surpassing Human Experts

During extensive experiments and training, the CUDA-L2 system learned many advanced optimization techniques, some even exceeding the conventional thinking of human experts. These techniques are like unique cooking tricks discovered by chefs in long-term practice, both practical and unexpected.

The system first learned to choose appropriate implementation methods based on different matrix sizes. For smaller matrices, it tends to use direct WMMA (Warp Matrix Multiply Accumulate) instructions, a method that is simple and direct, like not needing complex equipment when making simple dishes. For large matrices, the system tends to use the more complex CuTe abstraction framework, which allows for finer multi-stage pipeline optimization.

Especially interesting is that CUDA-L2 discovered a counter-intuitive optimization strategy: matrix zero-padding technology. Traditional GPU programming usually requires matrix dimensions to be divisible by block size to avoid the complexity of boundary handling. But CUDA-L2 found that sometimes intentionally choosing a block size that cannot be divided evenly, and then expanding the matrix through zero-padding, can actually achieve better performance.

Take a specific example: for matrix multiplication of 8192×512×2048, traditional methods would choose a block size that can divide 8192 evenly, such as 128. But CUDA-L2 chose a block size of 160, padding the matrix M dimension from 8192 to 8320. Although this increased calculation overhead by about 1.6%, the overall performance improved by 15.2%. This is like intentionally adding a little extra water when cooking; although it seems wasteful, it allows the ingredients to blend better, resulting in a better taste in the end.

In terms of memory access optimization, CUDA-L2 mastered advanced double-buffering technology. Traditional methods load data A first, then load data B, and finally perform calculations when executing matrix operations. But CUDA-L2 learned to use a ping-pong buffering strategy: allocate two sets of register spaces, and while using one set for calculation, the other set pre-loads data for the next round. This technology greatly improves the overlap between computation and memory access.

This page provides an editorial summary based on publicly available information. It is not a republished article. Use the source link below for the original report.