Local development of kernels

Introduction

kernel-builder builds kernels in a sandbox. This has various benefits, such as building kernels for a wide range of Torch versions, compatibility with older C library versions and avoiding accidental dependencies.

However, this is not ideal during kernel development, since language servers and IDEs do not interpret the build.toml file. As a result, code completion will typically not work. kernel-builder provides the kernel-builder utility to generate CMake files to build native code and setuptools files for building the kernel as a regular Python package. Since CMake and setuptools are widely supported by IDEs, this provides a much-improved development experience.

Generating a Python project with kernel-builder

kernel-builder can create CMake/Python project files for a kernel with a build.toml file. The create-pyproject command will create the files for the kernel in the current directory:

$ kernel-builder create-pyproject -f

The -f flag is optional and instructs kernel-builder to overwrite existing files.

It is recommended to do an editable install of the generated project into your Python virtual environment for development:

$ pip install wheel # Needed once to enable bdist_wheel.
$ pip install --no-build-isolation -e .

You can also create a Python project for a kernel in another directory:

$ kernel-builder create-pyproject -f path/to/kernel

Warnings:

• Kernels built in this way should not be published on the Kernel Hub. They do not fulfill the kernel requirements.
• Do not add the generated files to Git. kernel-builder has regular updates and you generally want to use files generated by the latest version.

See IDE Setup for wiring the generated project into VS Code with direnv.

Testing kernel builds before publishing

Once you have built a kernel with kernel builder, you may want to test it locally with software that uses get_kernel or LayerRepository before publishing. This can be done using the LOCAL_KERNELS variable, which maps a repository ID to a local kernel directory. For example, you could use the kernel in devel/activation for any use of the kernels-community/activation repository with:

$ LOCAL_KERNELS="kernels-community/activation=devel/activation" \
  python my_app.py

It is also possible to map multiple repositories to local kernel directories by separating the entries with a colon (:):

$ LOCAL_KERNELS="kernels-community/activation=devel/activation:kernels-community/flash-attn2=devel/flash-attn2" \
  python my_app.py