Skip to content
Snippets Groups Projects
Xuefeng Ding (IHEP)'s avatar
Xuefeng Ding authored
cf4530b4

scikit-build-cpp-python

Why and What

use scikit-build-core to setup a python-cmake project that can be installed with pip install -e .

Conclusion

  • package name has nothing to do with the import path. pacakge named my_package is installed with pip install ., uninstalled with pip uninstall my_package, but can be used as import kk.ss as ks.
  • installation of cpp files:
install(TARGET mylib LIBRARY DESTINATION .)

and set

[tool.scikit-build]
wheel.install-dir = "kk/ss"
  • installation of python files:
[tool.scikit-build.wheel.packages]
"mm/tt" = "python/myproj/simulation"

[tool.scikit-build]
wheel.exclude = ["CMakeLists.txt","*.cc","*.h"]

special variable

  • ${SKBUILD_PLATLIB_DIR}: The original platlib directory. Anything here goes directly to site-packages when a wheel is installed.
  • ${SKBUILD_DATA_DIR}: The data directory. Anything here goes to the root of the environment when a wheel is installed (use with care).
  • ${SKBUILD_HEADERS_DIR}: The header directory. Anything in here gets installed to Python's header directory.
  • ${SKBUILD_SCRIPTS_DIR}: The scripts directory. Anything placed in here will go to bin (Unix) or Scripts (Windows).
  • ${SKBUILD_METADATA_DIR}: The dist-info directory. Licenses go in the licenses subdirectory. _Note that CMake is not run in the prepare_metadata_\* hooks, so anything written to this directory will only be present when writing wheels._
  • ${SKBUILD_NULL_DIR}: Anything installed here will not be placed in the wheel.

style and lint check

notebook output stripping

  • use pre-commit
  1. install
pip install pre-commit nbstripout
  1. set
repos:
  - repo: https://gitlab.cern.ch/pre-commit-hook-mirrors/kynan/nbstripout
    rev: 0.8.0
    hooks:
      - id: nbstripout
        files: \.ipynb$
  1. install
pre-commit install
  1. run and verify
pre-commit run --all-files

or

nox -s pre_commit

c++ formating

  • pre-commit + vscode extensions
  • install extensions
Name: clangd
Id: llvm-vs-code-extensions.vscode-clangd
Description: C/C++ completion, navigation, and insights
Version: 0.1.30
Publisher: LLVM
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd

and make sure this is in settings.json:

  "[cpp]": {
    "editor.defaultFormatter": "llvm-vs-code-extensions.vscode-clangd"
  }
- repo: https://gitlab.cern.ch/pre-commit-hook-mirrors/pre-commit/mirrors-clang-format
  rev: v19.1.3
  hooks:
    - id: clang-format
      files: \.(cpp|hpp|c|h)$

python formatting

  • pre-commit + vscode extensions
  • install extensions
Name: Ruff
Id: charliermarsh.ruff
Description: A Visual Studio Code extension with support for the Ruff linter.
Version: 2024.56.0
Publisher: Astral Software
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff

and update settings.json

  "[python]": {
    "editor.codeActionsOnSave": {
      "source.fixAll": "explicit",
      "source.organizeImports": "explicit"
    },
    "editor.defaultFormatter": "charliermarsh.ruff"
  },
  "notebook.codeActionsOnSave": {
    "notebook.source.fixAll": "explicit",
    "notebook.source.organizeImports": "explicit"
  },
  "notebook.formatOnSave.enabled": true,
  • install pre-commit hooks
  - repo: https://github.com/charliermarsh/ruff-pre-commit
    rev: v0.7.4 # Use the latest version
    hooks:
      - id: ruff
        args: ["--fix", "--exit-non-zero-on-fix", "--show-fixes"]
      - id: ruff-formatp

c++ lint

  • nox + vscode extensions
  • setup nox
@nox.session(venv_backend="none")
def dev(session: nox.Session) -> None:
    """
    Prepare a .venv folder.
    """

    library_output_dir = DIR / "python" / "myproj" / "simulation" / "_csrc"

    session.run(sys.executable, "-m", "venv", ".venv")
    session.run(
        ".venv/bin/pip",
        "install",
        "scikit-build-core[pyproject]",
        "pybind11",
        "pip>=23",
        "nox",
        "pytest",
    )
    session.run(
        ".venv/bin/pip",
        "install",
        "--no-build-isolation",
        "--check-build-dependencies",
        "-ve",
        ".",
        "-Ccmake.define.CMAKE_EXPORT_COMPILE_COMMANDS=1",
        f"-Ccmake.define.CMAKE_LIBRARY_OUTPUT_DIRECTORY={library_output_dir}",
        "-Cbuild-dir=build",
    )

@nox.session(reuse_venv=True)
def cpp_lint(session: nox.Session) -> None:
    cpp_files = glob.glob("**/*.cc", recursive=True)
    session.install("clangd-tidy")
    session.run("clangd-tidy", "-p", "build", *cpp_files)
  • vscode extensions. use clangd
Name: clangd
Id: llvm-vs-code-extensions.vscode-clangd
Description: C/C++ completion, navigation, and insights
Version: 0.1.30
Publisher: LLVM
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd

and set

"clangd.arguments": [
    "--log=info",
    "--pretty",
    "--all-scopes-completion",
    "--completion-style=bundled",
    "--header-insertion=iwyu",
    "--header-insertion-decorators",
    "--background-index",
    "--clang-tidy",
    "-j=20",
    "--pch-storage=disk",
    "--function-arg-placeholders=false",
    "--compile-commands-dir=build"
  ],
"clangd.path": "clangd-17"

python lint

ruff: pre-commit + vscode extensions

  • install pre-commit
- repo: https://github.com/charliermarsh/ruff-pre-commit
  rev: v0.7.3 # Use the latest version
  hooks:
    - id: ruff
      args: ["--fix", "--show-fixes"]
  • install extensions
Name: Ruff
Id: charliermarsh.ruff
Description: A Visual Studio Code extension with support for the Ruff linter.
Version: 2024.54.0
Publisher: Astral Software
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff

pyright: pre-commit + vscode

  • install pre-commit
repos:
  - repo: https://github.com/RobertCraigie/pyright-python
    rev: v1.1.389
    hooks:
      - id: pyright
  • vscode: included in pylance automatically
{
  "python.defaultInterpreterPath": ".venv/bin/python",
  "python.analysis.autoFormatStrings": true,
  "python.analysis.autoImportCompletions": true,
  "python.analysis.completeFunctionParens": true,
  "python.analysis.inlayHints.functionReturnTypes": true,
  "python.analysis.inlayHints.pytestParameters": true,
  "python.analysis.inlayHints.variableTypes": true,
  "python.analysis.typeCheckingMode": "strict",
  "python.analysis.extraPaths": ["${workspaceFolder}/python"]
}

mypy pre-commit

  • install pre-commit
- repo: https://github.com/pre-commit/mirrors-mypy
  rev: "v1.13.0"
  hooks:
    - id: mypy

not used

  • pylint, replaced by ruff-lint
  • black, replaced by ruff-format
  • mypy, replaced by pyright

devcontainer

  • with podman, it can be run in rootless mode
  • podman + root inside and rootless (vscode) inside need different configurations