Skip to content
Snippets Groups Projects
README.md 6.72 KiB
Newer Older
Xuefeng Ding's avatar
Xuefeng Ding committed
# scikit-build-cpp-python

## Why and What
Xuefeng Ding's avatar
Xuefeng Ding committed

use scikit-build-core to setup a python-cmake project that can be installed with `pip install -e .`
Xuefeng Ding's avatar
Xuefeng Ding committed

## Conclusion

Xuefeng Ding's avatar
Xuefeng Ding committed
- package name has nothing to do with the import path. pacakge named my_package is installed with `pip install .`, uninstalled with `pip uninstall my_package`, but can be used as `import kk.ss as ks`.
Xuefeng Ding's avatar
Xuefeng Ding committed
- installation of cpp files:
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```cmake
Xuefeng Ding's avatar
Xuefeng Ding committed
install(TARGET mylib LIBRARY DESTINATION .)
Xuefeng Ding's avatar
Xuefeng Ding committed
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
and set
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```
[tool.scikit-build]
wheel.install-dir = "kk/ss"
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- installation of python files:
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```
[tool.scikit-build.wheel.packages]
"mm/tt" = "python/myproj/simulation"

[tool.scikit-build]
wheel.exclude = ["CMakeLists.txt","*.cc","*.h"]
```
Xuefeng Ding's avatar
Xuefeng Ding committed

## special variable

- `${SKBUILD_PLATLIB_DIR}`: The original platlib directory. Anything here goes
  directly to site-packages when a wheel is installed.
- `${SKBUILD_DATA_DIR}`: The data directory. Anything here goes to the root of
  the environment when a wheel is installed (use with care).
- `${SKBUILD_HEADERS_DIR}`: The header directory. Anything in here gets
  installed to Python's header directory.
- `${SKBUILD_SCRIPTS_DIR}`: The scripts directory. Anything placed in here will
  go to `bin` (Unix) or `Scripts` (Windows).
- `${SKBUILD_METADATA_DIR}`: The dist-info directory. Licenses go in the
  `licenses` subdirectory. _Note that CMake is not run in the
  `prepare_metadata_\*` hooks, so anything written to this directory will only
  be present when writing wheels.\_
- `${SKBUILD_NULL_DIR}`: Anything installed here will not be placed in the
  wheel.

Xuefeng Ding's avatar
Xuefeng Ding committed
## style and lint check
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
### notebook output stripping
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- use pre-commit
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
1. install
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```bash
pip install pre-commit nbstripout
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
2. set
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```yaml
repos:
Xuefeng Ding's avatar
Xuefeng Ding committed
  - repo: https://gitlab.cern.ch/pre-commit-hook-mirrors/kynan/nbstripout
    rev: 0.8.0
Xuefeng Ding's avatar
Xuefeng Ding committed
    hooks:
      - id: nbstripout
        files: \.ipynb$
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
3. install
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```bash
pre-commit install
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
4. run and verify
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```bash
pre-commit run --all-files
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
or
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```bash
nox -s pre_commit
```

### c++ formating
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- pre-commit + vscode extensions
- install extensions
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```
Name: clangd
Id: llvm-vs-code-extensions.vscode-clangd
Description: C/C++ completion, navigation, and insights
Version: 0.1.30
Publisher: LLVM
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
and make sure this is in `settings.json`:
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```json
  "[cpp]": {
    "editor.defaultFormatter": "llvm-vs-code-extensions.vscode-clangd"
  }
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- install pre-commit hooks
Xuefeng Ding's avatar
Xuefeng Ding committed
  Name: Black Formatter
  Id: ms-python.black-formatter
  Description: Formatting support for Python files using the Black formatter.
  Version: 2024.4.0
  Publisher: Microsoft
  VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-python.black-formatter

Xuefeng Ding's avatar
Xuefeng Ding committed
```yaml
- repo: https://gitlab.cern.ch/pre-commit-hook-mirrors/pre-commit/mirrors-clang-format
  rev: v19.1.3
  hooks:
    - id: clang-format
      files: \.(cpp|hpp|c|h)$
```

### python formatting
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- pre-commit + vscode extensions
- install extensions
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```
Xuefeng Ding's avatar
Xuefeng Ding committed
Name: Ruff
Id: charliermarsh.ruff
Description: A Visual Studio Code extension with support for the Ruff linter.
Version: 2024.56.0
Publisher: Astral Software
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff
Xuefeng Ding's avatar
Xuefeng Ding committed
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
and update `settings.json`
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```json
  "[python]": {
    "editor.codeActionsOnSave": {
Xuefeng Ding's avatar
Xuefeng Ding committed
      "source.fixAll": "explicit",
      "source.organizeImports": "explicit"
Xuefeng Ding's avatar
Xuefeng Ding committed
    },
Xuefeng Ding's avatar
Xuefeng Ding committed
    "editor.defaultFormatter": "charliermarsh.ruff"
Xuefeng Ding's avatar
Xuefeng Ding committed
  },
Xuefeng Ding's avatar
Xuefeng Ding committed
  "notebook.codeActionsOnSave": {
    "notebook.source.fixAll": "explicit",
    "notebook.source.organizeImports": "explicit"
  },
  "notebook.formatOnSave.enabled": true,
Xuefeng Ding's avatar
Xuefeng Ding committed
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- install pre-commit hooks
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```
  - repo: https://github.com/charliermarsh/ruff-pre-commit
    rev: v0.7.4 # Use the latest version
Xuefeng Ding's avatar
Xuefeng Ding committed
    hooks:
Xuefeng Ding's avatar
Xuefeng Ding committed
      - id: ruff
        args: ["--fix", "--exit-non-zero-on-fix", "--show-fixes"]
      - id: ruff-formatp
Xuefeng Ding's avatar
Xuefeng Ding committed
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
### c++ lint
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- nox + vscode extensions
Xuefeng Ding's avatar
Xuefeng Ding committed
- setup nox
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```python
@nox.session(venv_backend="none")
def dev(session: nox.Session) -> None:
    """
    Prepare a .venv folder.
    """

Xuefeng Ding's avatar
Xuefeng Ding committed
    library_output_dir = DIR / "python" / "myproj" / "simulation" / "_csrc"

Xuefeng Ding's avatar
Xuefeng Ding committed
    session.run(sys.executable, "-m", "venv", ".venv")
    session.run(
        ".venv/bin/pip",
        "install",
        "scikit-build-core[pyproject]",
        "pybind11",
        "pip>=23",
Xuefeng Ding's avatar
Xuefeng Ding committed
        "nox",
        "pytest",
Xuefeng Ding's avatar
Xuefeng Ding committed
    )
    session.run(
        ".venv/bin/pip",
        "install",
        "--no-build-isolation",
        "--check-build-dependencies",
Xuefeng Ding's avatar
Xuefeng Ding committed
        "-ve",
        ".",
        "-Ccmake.define.CMAKE_EXPORT_COMPILE_COMMANDS=1",
        f"-Ccmake.define.CMAKE_LIBRARY_OUTPUT_DIRECTORY={library_output_dir}",
        "-Cbuild-dir=build",
Xuefeng Ding's avatar
Xuefeng Ding committed
    )

@nox.session(reuse_venv=True)
def cpp_lint(session: nox.Session) -> None:
    cpp_files = glob.glob("**/*.cc", recursive=True)
    session.install("clangd-tidy")
    session.run("clangd-tidy", "-p", "build", *cpp_files)
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- vscode extensions. use `clangd`
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```
Name: clangd
Id: llvm-vs-code-extensions.vscode-clangd
Description: C/C++ completion, navigation, and insights
Version: 0.1.30
Publisher: LLVM
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
and set
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```json
"clangd.arguments": [
    "--log=info",
    "--pretty",
    "--all-scopes-completion",
    "--completion-style=bundled",
    "--header-insertion=iwyu",
    "--header-insertion-decorators",
    "--background-index",
    "--clang-tidy",
    "-j=20",
    "--pch-storage=disk",
    "--function-arg-placeholders=false",
    "--compile-commands-dir=build"
  ],
"clangd.path": "clangd-17"
```
Xuefeng Ding's avatar
Xuefeng Ding committed

### python lint
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
#### ruff: pre-commit + vscode extensions
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- install pre-commit
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```yaml
Xuefeng Ding's avatar
Xuefeng Ding committed
- repo: https://github.com/charliermarsh/ruff-pre-commit
  rev: v0.7.3 # Use the latest version
  hooks:
    - id: ruff
      args: ["--fix", "--show-fixes"]
Xuefeng Ding's avatar
Xuefeng Ding committed
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- install extensions
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```
Name: Ruff
Id: charliermarsh.ruff
Description: A Visual Studio Code extension with support for the Ruff linter.
Version: 2024.54.0
Publisher: Astral Software
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
#### pylint: nox + vscode extensions
Xuefeng Ding's avatar
Xuefeng Ding committed

## vscode extensions to install
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- "isort", which sort python import
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```
Name: isort
Id: ms-python.isort
Description: Import organization support for Python files using isort.
Version: 2023.10.1
Publisher: Microsoft
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-python.isort
```
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
- "pylint", for complrehensive code analysis
Xuefeng Ding's avatar
Xuefeng Ding committed

Xuefeng Ding's avatar
Xuefeng Ding committed
```
Name: Pylint
Id: ms-python.pylint
Description: Linting support for Python files using Pylint.
Version: 2024.0.0
Publisher: Microsoft
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-python.pylint
```