ci: Updated to checkout@v3

disabled components in builder.py

.github/workflows/checks.yml

@@ -0,0 +1,40 @@
+name: Checks
+
+on:
+  pull_request:
+  push:
+    branches:
+      - stable
+      - dev
+
+jobs:
+  checks:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v3
+
+    - name: Set up Python
+      uses: actions/setup-python@v3
+      with:
+        python-version: '3.11'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install -r requirements.txt
+        python -m pip install mypy pytest ruff
+
+    - name: Type Checking
+      run: |
+        python -m mypy --install-types --non-interactive .
+
+    - name: Linting
+      run: |
+        ruff check cutekit
+
+    - name: Unit Testing
+      run: |
+        python -m pytest
+

.github/workflows/publish.yml

@@ -17,15 +17,17 @@ permissions:
 
 jobs:
   deploy:
-
     runs-on: ubuntu-latest
 
     steps:
-    - uses: actions/checkout@v3
+    - name: Checkout repository
+      uses: actions/checkout@v3
+
     - name: Set up Python
       uses: actions/setup-python@v3
       with:
-        python-version: '3.x'
+        python-version: '3.11'
+
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip

LICENSE

@@ -1,6 +1,7 @@
 MIT License
 
-Copyright (c) 2022 skift
+Copyright (c) 2022, the skiftOS Developers
+Copyright (c) 2022-2023, Cute Engineering
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1,257 +1,66 @@
-# osdk
-
-The operating system development kit 
+<br/>
+<br/>
+<br/>
+<p align="center">
+    <img src="logo.png" width="200" height="200">
+</p>
+<h1 align="center">CuteKit</h1>
+<p align="center">
+    The *magical* build system and package manager
+</p>
+<br/>
+<br/>
+<br/>
 
 ## Table of contents
 
-- [osdk](#osdk)
-  - [Table of contents](#table-of-contents)
-  - [Macros](#macros)
-    - [`@latest`](#latest)
-    - [`@uname`](#uname)
-    - [`@include`](#include)
-    - [`@join`](#join)
-    - [`@concat`](#concat)
-    - [`@exec`](#exec)
-  - [Manifest file format](#manifest-file-format)
-    - [`id`](#id)
-    - [`type`](#type)
-    - [`description`](#description)
-    - [`enabledIf`](#enabledif)
-    - [`requires`](#requires)
-    - [`provides`](#provides)
-  - [Target file format](#target-file-format)
-    - [`id`](#id-1)
-    - [`type`](#type-1)
-    - [`props`](#props)
-    - [`tools`](#tools)
+- [Table of contents](#table-of-contents)
+- [Introduction](#introduction)
+- [Quick-start](#quick-start)
+- [Example](#example)
 
 
-## Macros
+## Introduction
 
+**CuteKit** is a simple - yet - powerful build system and package manager for C and C++. It:
 
-### `@latest`
+- ✨ It uses **JSON**: Cutekit uses JSON instead of introducing a whole new programming language for describing the project. And also has macros to help the user experience (see [Jexpr](doc/spec/jexpr.md)).
+- ✨ It's a **package manager**: Cutekit package manager is based on **Git**, nothing is centralized.
+- ✨ It's **extendible**: Cutekit can be [extended](./doc/extends.md) by writing custom Python plugins.
+- ✨ It's **easy**: the [**templates**](./doc/templates.md) help the user quick-start a project.
+- ✨ It's **portable**: Cutekit can run on MacOS Gnu/Linux and Windows.
 
-Find the latest version of a command in the path.
+## Installation
 
-```json
-"cc": {
-    "cmd": ["@latest", "clang"], // clang-14
-/* ... */
+To install Cutekit, you may use your favourite package manager if it is available. Or you can install it manually by following the instructions below.
+
+```bash
+$ git clone https://github.com/cute-engineering/cutekit
+
+$ cd cutekit
+
+# If you want to use the latest version of Cutekit, you can switch to the dev branch.
+# $ git switch dev
+
+$ pip install --user -e .
 ```
 
-### `@uname`
+## Quick-start
 
-Query the system for information about the current operating system.
+-> If you directly want to start using Cutekit for a new project, you can just run `$ ck I host` and it will create a new project in the host directory (you can rename it later).
 
+-> If you want to use Cutekit for writing operating systems, you can create a new [limine](https://github.com/limine-bootloader/limine/)-based project by running `$ ck I limine-barebone`.
 
-```json
-"cc": {
-    "cmd": ["@uname", "machine"], // "x86_64"
-/* ... */
-```
+## Example
 
-### `@include`
+If you want to see how it works you can read the [doc/cutekit.md](doc/cutekit.md) file.
 
-Include a file
+# License
 
-### `@join`
+<a href="https://opensource.org/licenses/MIT">
+  <img align="right" height="96" alt="MIT License" src="doc/mit.svg" />
+</a>
 
-Join two objects
+Cutekit is licensed under the **MIT License**.
 
-Example:
-
-```json
-["@join",  {"a": 1}, {"b": 2}] // {"a": 1, "b": 2}
-```
-
-### `@concat`
-
-Concatenate strings
-
-Example:
-
-```json
-["@concat", "a", "b", "c"] // "abc"
-```
-
-### `@exec`
-
-Execute a command and return the output
-
-Example:
-
-```json
-["@exec", "uname", "-m"] // "x86_64"
-```
-
-## Manifest file format
-
-### `id`
-
-The id of the package. This is used to identify the package in the manifest file.
-
-Exemple:
-
-```json
-{
-    "id": "hello"
-}
-```
-
-### `type`
-
-The type of the package. This is used to identify the package in the manifest file.
-
-Exemple:
-
-```json
-{
-    "type": "exe"
-}
-```
-
-### `description`
-
-The description of the package for the user.
-
-Exemple:
-
-```json
-{
-    "description": "Hello world"
-}
-```
-
-### `enabledIf`
-
-A list of requirements for the package check agaisnt the build props. If the requirement is not met, the package will be disabled.
-
-```json
-{
-    "enabledIf": {
-        "freestanding": [
-            false
-        ]
-    }
-}
-```
-
-### `requires`
-
-Dependencies of the package. The name listed here must be the same as the id of the package or member of a provide list.
-
-Exemple:
-
-```json
-{
-    "requires": [
-        "libc",
-        "libm"
-    ]
-}
-```
-
-### `provides`
-
-Alias for the package.
-
-Exemple:
-
-```json
-{
-    "provides": [
-        "hello"
-    ]
-}
-```
-
-
-## Target file format
-
-### `id`
-
-The id of the target. This is used to identify the target in the target file.
-
-### `type`
-
-Should be `target`.
-
-### `props`
-
-A list of properties for the target.
-
-Exemple:
-
-```json
-{
-    "props": {
-        "arch": "x86_64",
-        "vendor": "pc",
-        "os": "linux",
-        "env": "gnu",
-        "abi": "elf",
-        "cpu": "x86_64",
-        "features": "fxsr,sse,sse2"
-    }
-}
-```
-
-Theses values are exposed the translation unit as `__osdk_{prop}__`.
-
-### `tools`
-
-A list of tools for the target.
-
-```json
-{
-    "tools": {
-        "cc": {
-            "cmd": ["@latest", "clang"],
-            "args": [
-                "-target",
-                "x86_64-unknown-windows",
-                "-ffreestanding",
-                "-fno-stack-protector",
-                "-fshort-wchar",
-                "-mno-red-zone"
-            ]
-        },
-        "cxx": {
-            "cmd": ["@latest", "clang++"],
-            "args": [
-                "-target",
-                "x86_64-unknown-windows",
-                "-ffreestanding",
-                "-fno-stack-protector",
-                "-fshort-wchar",
-                "-mno-red-zone"
-            ]
-        },
-        "ld": {
-            "cmd": ["@latest", "clang++"],
-            "args": [
-                "-target",
-                "x86_64-unknown-windows",
-                "-nostdlib",
-                "-Wl,-entry:efi_main",
-                "-Wl,-subsystem:efi_application",
-                "-fuse-ld=lld-link"
-            ]
-        },
-        "ar": {
-            "cmd": ["@latest", "llvm-ar"],
-            "args": [
-                "rcs"
-            ]
-        },
-        "as": {
-            "cmd": "nasm",
-            "args": [
-                "-f",
-                "win64"
-            ]
-        }
-    }
-}
-```
+The full text of the license can be accessed via [this link](https://opensource.org/licenses/MIT) and is also included in the [license.md](license.md) file of this software package.

cutekit/__init__.py

@@ -0,0 +1,72 @@
+import sys
+import os
+import logging
+
+from . import (
+    builder,  # noqa: F401 this is imported for side effects
+    cli,
+    const,
+    graph,  # noqa: F401 this is imported for side effects
+    model,
+    plugins,
+    vt100,
+)
+
+
+def ensure(version: tuple[int, int, int]):
+    if (
+        const.VERSION[0] == version[0]
+        and const.VERSION[1] == version[1]
+        and const.VERSION[2] >= version[2]
+    ):
+        return
+
+    raise RuntimeError(
+        f"Expected cutekit version {version[0]}.{version[1]}.{version[2]} but found {const.VERSION_STR}"
+    )
+
+
+def setupLogger(verbose: bool):
+    if verbose:
+        logging.basicConfig(
+            level=logging.DEBUG,
+            format=f"{vt100.CYAN}%(asctime)s{vt100.RESET} {vt100.YELLOW}%(levelname)s{vt100.RESET} %(name)s: %(message)s",
+            datefmt="%Y-%m-%d %H:%M:%S",
+        )
+    else:
+        projectRoot = model.Project.topmost()
+        logFile = const.GLOBAL_LOG_FILE
+        if projectRoot is not None:
+            logFile = os.path.join(projectRoot.dirname(), const.PROJECT_LOG_FILE)
+
+        # create the directory if it doesn't exist
+        logDir = os.path.dirname(logFile)
+        if not os.path.isdir(logDir):
+            os.makedirs(logDir)
+
+        logging.basicConfig(
+            level=logging.INFO,
+            filename=logFile,
+            filemode="w",
+            format="%(asctime)s %(levelname)s %(name)s: %(message)s",
+            datefmt="%Y-%m-%d %H:%M:%S",
+        )
+
+
+def main() -> int:
+    try:
+        a = cli.parse(sys.argv[1:])
+        setupLogger(a.consumeOpt("verbose", False) is True)
+        plugins.loadAll()
+        cli.exec(a)
+        print()
+        return 0
+    except RuntimeError as e:
+        logging.exception(e)
+        cli.error(str(e))
+        cli.usage()
+        print()
+        return 1
+    except KeyboardInterrupt:
+        print()
+        return 1

cutekit/builder.py

@@ -0,0 +1,298 @@
+import os
+import logging
+import dataclasses as dt
+
+from pathlib import Path
+from typing import TextIO, Union
+
+from . import shell, rules, model, ninja, const, cli
+
+_logger = logging.getLogger(__name__)
+
+
+def aggregateCincs(target: model.Target, registry: model.Registry) -> set[str]:
+    res = set()
+
+    for c in registry.iterEnabled(target):
+        if "cpp-root-include" in c.props:
+            res.add(c.dirname())
+        elif c.type == model.Kind.LIB:
+            res.add(str(Path(c.dirname()).parent))
+
+    return set(map(lambda i: f"-I{i}", res))
+
+
+def aggregateCdefs(target: model.Target) -> set[str]:
+    res = set()
+
+    def sanatize(s: str) -> str:
+        return s.lower().replace(" ", "_").replace("-", "_").replace(".", "_")
+
+    for k, v in target.props.items():
+        if isinstance(v, bool):
+            if v:
+                res.add(f"-D__ck_{sanatize(k)}__")
+        else:
+            res.add(f"-D__ck_{sanatize(k)}_{sanatize(str(v))}__")
+            res.add(f"-D__ck_{sanatize(k)}_value={str(v)}")
+
+    return res
+
+
+def buildpath(target: model.Target, component: model.Component, path) -> Path:
+    return Path(target.builddir) / component.id / path
+
+
+# --- Compilation ------------------------------------------------------------ #
+
+
+def wilcard(component: model.Component, wildcards: list[str]) -> list[str]:
+    dirs = [component.dirname()] + list(
+        map(lambda d: os.path.join(component.dirname(), d), component.subdirs)
+    )
+    return shell.find(dirs, list(wildcards), recusive=False)
+
+
+def compile(
+    w: ninja.Writer,
+    target: model.Target,
+    component: model.Component,
+    rule: str,
+    srcs: list[str],
+) -> list[str]:
+    res: list[str] = []
+    for src in srcs:
+        rel = Path(src).relative_to(component.dirname())
+        dest = buildpath(target, component, "obj") / rel.with_suffix(".o")
+        t = target.tools[rule]
+        w.build(str(dest), rule, inputs=src, order_only=t.files)
+        res.append(str(dest))
+    return res
+
+
+# --- Ressources ------------------------------------------------------------- #
+
+
+def listRes(component: model.Component) -> list[str]:
+    return shell.find(str(component.subpath("res")))
+
+
+def compileRes(
+    w: ninja.Writer,
+    target: model.Target,
+    component: model.Component,
+) -> list[str]:
+    res: list[str] = []
+    for r in listRes(component):
+        rel = Path(r).relative_to(component.subpath("res"))
+        dest = buildpath(target, component, "res") / rel
+        w.build(str(dest), "cp", r)
+        res.append(str(dest))
+    return res
+
+
+# --- Linking ---------------------------------------------------------------- #
+
+
+def outfile(target: model.Target, component: model.Component) -> str:
+    if component.type == model.Kind.LIB:
+        return str(buildpath(target, component, f"lib/{component.id}.a"))
+    else:
+        return str(buildpath(target, component, f"bin/{component.id}.out"))
+
+
+def collectLibs(
+    registry: model.Registry, target: model.Target, component: model.Component
+) -> list[str]:
+    res: list[str] = []
+    for r in component.resolved[target.id].resolved:
+        req = registry.lookup(r, model.Component)
+        assert req is not None  # model.Resolver has already checked this
+
+        if r == component.id:
+            continue
+        if not req.type == model.Kind.LIB:
+            raise RuntimeError(f"Component {r} is not a library")
+        res.append(outfile(target, req))
+    return res
+
+
+def link(
+    w: ninja.Writer,
+    registry: model.Registry,
+    target: model.Target,
+    component: model.Component,
+) -> str:
+    w.newline()
+    out = outfile(target, component)
+
+    objs = []
+    objs += compile(w, target, component, "cc", wilcard(component, ["*.c"]))
+    objs += compile(
+        w, target, component, "cxx", wilcard(component, ["*.cpp", "*.cc", "*.cxx"])
+    )
+    objs += compile(
+        w, target, component, "as", wilcard(component, ["*.s", "*.asm", "*.S"])
+    )
+
+    res = compileRes(w, target, component)
+    libs = collectLibs(registry, target, component)
+    if component.type == model.Kind.LIB:
+        w.build(out, "ar", objs, implicit=res)
+    else:
+        w.build(out, "ld", objs + libs, implicit=res)
+    return out
+
+
+# --- Phony ------------------------------------------------------------------ #
+
+
+def all(w: ninja.Writer, registry: model.Registry, target: model.Target) -> list[str]:
+    all: list[str] = []
+    for c in registry.iterEnabled(target):
+        all.append(link(w, registry, target, c))
+    w.build("all", "phony", all)
+    w.default("all")
+    return all
+
+
+def gen(out: TextIO, target: model.Target, registry: model.Registry):
+    w = ninja.Writer(out)
+
+    w.comment("File generated by the build system, do not edit")
+    w.newline()
+
+    w.variable("builddir", target.builddir)
+    w.variable("hashid", target.hashid)
+
+    w.separator("Tools")
+
+    w.variable("cincs", " ".join(aggregateCincs(target, registry)))
+    w.variable("cdefs", " ".join(aggregateCdefs(target)))
+    w.newline()
+
+    for i in target.tools:
+        tool = target.tools[i]
+        rule = rules.rules[i]
+        w.variable(i, tool.cmd)
+        w.variable(i + "flags", " ".join(rule.args + tool.args))
+        w.rule(
+            i,
+            f"{tool.cmd} {rule.rule.replace('$flags',f'${i}flags')}",
+            depfile=rule.deps,
+        )
+        w.newline()
+
+    w.separator("Build")
+
+    all(w, registry, target)
+
+
+@dt.dataclass
+class Product:
+    path: Path
+    target: model.Target
+    component: model.Component
+
+
+def build(
+    target: model.Target,
+    registry: model.Registry,
+    components: Union[list[model.Component], model.Component, None] = None,
+) -> list[Product]:
+    all = False
+    shell.mkdir(target.builddir)
+    ninjaPath = os.path.join(target.builddir, "build.ninja")
+
+    if not os.path.exists(ninjaPath):
+        with open(ninjaPath, "w") as f:
+            gen(f, target, registry)
+
+    if components is None:
+        all = True
+        components = list(registry.iterEnabled(target))
+
+    if isinstance(components, model.Component):
+        components = [components]
+
+    products: list[Product] = []
+    for c in components:
+        r = c.resolved[target.id]
+        if not r.enabled:
+            raise RuntimeError(f"Component {c.id} is disabled: {r.reason}")
+
+        products.append(
+            Product(
+                path=Path(outfile(target, c)),
+                target=target,
+                component=c,
+            )
+        )
+
+    outs = list(map(lambda p: str(p.path), products))
+
+    shell.exec("ninja", "-f", ninjaPath, *(outs if not all else []))
+
+    return products
+
+
+# --- Commands --------------------------------------------------------------- #
+
+
+@cli.command("b", "build", "Build a component or all components")
+def buildCmd(args: cli.Args):
+    registry = model.Registry.use(args)
+    target = model.Target.use(args)
+    componentSpec = args.consumeArg()
+    component = None
+    if componentSpec is not None:
+        component = registry.lookup(componentSpec, model.Component)
+    build(target, registry, component)[0]
+
+
+@cli.command("r", "run", "Run a component")
+def runCmd(args: cli.Args):
+    registry = model.Registry.use(args)
+    target = model.Target.use(args)
+    debug = args.consumeOpt("debug", False) is True
+
+    componentSpec = args.consumeArg() or "__main__"
+    component = registry.lookup(componentSpec, model.Component, includeProvides=True)
+    if component is None:
+        raise RuntimeError(f"Component {componentSpec} not found")
+
+    product = build(target, registry, component)[0]
+
+    os.environ["CK_TARGET"] = target.id
+    os.environ["CK_COMPONENT"] = product.component.id
+    os.environ["CK_BUILDDIR"] = target.builddir
+
+    shell.exec(*(["lldb", "-o", "run"] if debug else []), str(product.path), *args.args)
+
+
+@cli.command("t", "test", "Run all test targets")
+def testCmd(args: cli.Args):
+    # This is just a wrapper around the `run` command that try
+    # to run a special hook component named __tests__.
+    args.args.insert(0, "__tests__")
+    runCmd(args)
+
+
+@cli.command("d", "debug", "Debug a component")
+def debugCmd(args: cli.Args):
+    # This is just a wrapper around the `run` command that
+    # always enable debug mode.
+    args.opts["debug"] = True
+    runCmd(args)
+
+
+@cli.command("c", "clean", "Clean build files")
+def cleanCmd(args: cli.Args):
+    model.Project.use(args)
+    shell.rmrf(const.BUILD_DIR)
+
+
+@cli.command("n", "nuke", "Clean all build files and caches")
+def nukeCmd(args: cli.Args):
+    model.Project.use(args)
+    shell.rmrf(const.PROJECT_CK_DIR)

cutekit/cli.py

@@ -0,0 +1,165 @@
+import inspect
+import logging
+import sys
+import dataclasses as dt
+
+from pathlib import Path
+from typing import Optional, Union, Callable
+
+from . import const, vt100
+
+Value = Union[str, bool, int]
+
+_logger = logging.getLogger(__name__)
+
+
+class Args:
+    opts: dict[str, Value]
+    args: list[str]
+
+    def __init__(self):
+        self.opts = {}
+        self.args = []
+
+    def consumePrefix(self, prefix: str) -> dict[str, Value]:
+        result: dict[str, Value] = {}
+        copy = self.opts.copy()
+        for key, value in copy.items():
+            if key.startswith(prefix):
+                result[key[len(prefix) :]] = value
+                del self.opts[key]
+        return result
+
+    def consumeOpt(self, key: str, default: Value = False) -> Value:
+        if key in self.opts:
+            result = self.opts[key]
+            del self.opts[key]
+            return result
+        return default
+
+    def tryConsumeOpt(self, key: str) -> Optional[Value]:
+        if key in self.opts:
+            result = self.opts[key]
+            del self.opts[key]
+            return result
+        return None
+
+    def consumeArg(self, default: Optional[str] = None) -> Optional[str]:
+        if len(self.args) == 0:
+            return default
+
+        first = self.args[0]
+        del self.args[0]
+        return first
+
+
+def parse(args: list[str]) -> Args:
+    result = Args()
+
+    for arg in args:
+        if arg.startswith("--"):
+            if "=" in arg:
+                key, value = arg[2:].split("=", 1)
+                result.opts[key] = value
+            else:
+                result.opts[arg[2:]] = True
+        else:
+            result.args.append(arg)
+
+    return result
+
+
+Callback = Callable[[Args], None]
+
+
+@dt.dataclass
+class Command:
+    shortName: Optional[str]
+    longName: str
+    helpText: str
+    isPlugin: bool
+    callback: Callback
+
+
+commands: list[Command] = []
+
+
+def command(shortName: Optional[str], longName: str, helpText: str):
+    curframe = inspect.currentframe()
+    calframe = inspect.getouterframes(curframe, 2)
+
+    def wrap(fn: Callable[[Args], None]):
+        _logger.debug(f"Registering command {longName}")
+        commands.append(
+            Command(
+                shortName,
+                longName,
+                helpText,
+                Path(calframe[1].filename).parent != Path(__file__).parent,
+                fn,
+            )
+        )
+        return fn
+
+    return wrap
+
+
+# --- Builtins Commands ------------------------------------------------------ #
+
+
+@command("u", "usage", "Show usage information")
+def usage(args: Optional[Args] = None):
+    print(f"Usage: {const.ARGV0} <command> [args...]")
+
+
+def error(msg: str) -> None:
+    print(f"{vt100.RED}Error:{vt100.RESET} {msg}\n", file=sys.stderr)
+
+
+@command("h", "help", "Show this help message")
+def helpCmd(args: Args):
+    usage()
+
+    print()
+
+    vt100.title("Description")
+    print(f"    {const.DESCRIPTION}")
+
+    print()
+    vt100.title("Commands")
+    for cmd in sorted(commands, key=lambda c: c.longName):
+        if cmd.longName.startswith("_"):
+            continue
+
+        pluginText = ""
+        if cmd.isPlugin:
+            pluginText = f"{vt100.CYAN}(plugin){vt100.RESET}"
+
+        print(
+            f" {vt100.GREEN}{cmd.shortName or ' '}{vt100.RESET}  {cmd.longName} - {cmd.helpText} {pluginText}"
+        )
+
+    print()
+    vt100.title("Logging")
+    print("    Logs are stored in:")
+    print(f"     - {const.PROJECT_LOG_FILE}")
+    print(f"     - {const.GLOBAL_LOG_FILE}")
+
+
+@command("v", "version", "Show current version")
+def versionCmd(args: Args):
+    print(f"CuteKit v{const.VERSION_STR}")
+
+
+def exec(args: Args):
+    cmd = args.consumeArg()
+
+    if cmd is None:
+        raise RuntimeError("No command specified")
+
+    for c in commands:
+        if c.shortName == cmd or c.longName == cmd:
+            c.callback(args)
+            return
+
+    raise RuntimeError(f"Unknown command {cmd}")

cutekit/compat.py

@@ -0,0 +1,37 @@
+from pathlib import Path
+from typing import Any
+
+
+SUPPORTED_MANIFEST = [
+    "https://schemas.cute.engineering/stable/cutekit.manifest.component.v1",
+    "https://schemas.cute.engineering/stable/cutekit.manifest.project.v1",
+    "https://schemas.cute.engineering/stable/cutekit.manifest.target.v1",
+]
+
+OSDK_MANIFEST_NOT_SUPPORTED = (
+    "OSDK manifests are not supported by CuteKit. Please use CuteKit manifest instead"
+)
+
+UNSUPORTED_MANIFEST = {
+    "https://schemas.cute.engineering/stable/osdk.manifest.component.v1": OSDK_MANIFEST_NOT_SUPPORTED,
+    "https://schemas.cute.engineering/stable/osdk.manifest.project.v1": OSDK_MANIFEST_NOT_SUPPORTED,
+    "https://schemas.cute.engineering/stable/osdk.manifest.target.v1": OSDK_MANIFEST_NOT_SUPPORTED,
+    "https://schemas.cute.engineering/latest/osdk.manifest.component": OSDK_MANIFEST_NOT_SUPPORTED,
+    "https://schemas.cute.engineering/latest/osdk.manifest.project": OSDK_MANIFEST_NOT_SUPPORTED,
+    "https://schemas.cute.engineering/latest/osdk.manifest.target": OSDK_MANIFEST_NOT_SUPPORTED,
+}
+
+
+def ensureSupportedManifest(manifest: Any, path: Path):
+    if "$schema" not in manifest:
+        raise RuntimeError(f"Missing $schema in {path}")
+
+    if manifest["$schema"] in UNSUPORTED_MANIFEST:
+        raise RuntimeError(
+            f"Unsupported manifest schema {manifest['$schema']} in {path}: {UNSUPORTED_MANIFEST[manifest['$schema']]}"
+        )
+
+    if manifest["$schema"] not in SUPPORTED_MANIFEST:
+        raise RuntimeError(
+            f"Unsupported manifest schema {manifest['$schema']} in {path}"
+        )

cutekit/const.py

@@ -0,0 +1,19 @@
+import os
+import sys
+
+VERSION = (0, 6, 0, "dev")
+VERSION_STR = f"{VERSION[0]}.{VERSION[1]}.{VERSION[2]}{'-' + VERSION[3] if len(VERSION) >= 4 else ''}"
+MODULE_DIR = os.path.dirname(os.path.realpath(__file__))
+ARGV0 = os.path.basename(sys.argv[0])
+PROJECT_CK_DIR = ".cutekit"
+GLOBAL_CK_DIR = os.path.join(os.path.expanduser("~"), ".cutekit")
+BUILD_DIR = os.path.join(PROJECT_CK_DIR, "build")
+CACHE_DIR = os.path.join(PROJECT_CK_DIR, "cache")
+EXTERN_DIR = os.path.join(PROJECT_CK_DIR, "extern")
+SRC_DIR = "src"
+META_DIR = "meta"
+TARGETS_DIR = os.path.join(META_DIR, "targets")
+DEFAULT_REPO_TEMPLATES = "cute-engineering/cutekit-templates"
+DESCRIPTION = "A build system and package manager for low-level software development"
+PROJECT_LOG_FILE = os.path.join(PROJECT_CK_DIR, "cutekit.log")
+GLOBAL_LOG_FILE = os.path.join(os.path.expanduser("~"), ".cutekit", "cutekit.log")

cutekit/graph.py

@@ -0,0 +1,95 @@
+import os
+
+from typing import Optional, cast
+
+from . import vt100, cli, model
+
+
+def view(
+    registry: model.Registry,
+    target: model.Target,
+    scope: Optional[str] = None,
+    showExe: bool = True,
+    showDisabled: bool = False,
+):
+    from graphviz import Digraph  # type: ignore
+
+    g = Digraph(target.id, filename="graph.gv")
+
+    g.attr("graph", splines="ortho", rankdir="BT", ranksep="1.5")
+    g.attr("node", shape="ellipse")
+    g.attr(
+        "graph",
+        label=f"<<B>{scope or 'Full Dependency Graph'}</B><BR/>{target.id}>",
+        labelloc="t",
+    )
+
+    scopeInstance = None
+
+    if scope is not None:
+        scopeInstance = registry.lookup(scope, model.Component)
+
+    for component in registry.iterEnabled(target):
+        if not component.type == model.Kind.LIB and not showExe:
+            continue
+
+        if (
+            scopeInstance is not None
+            and component.id != scope
+            and component.id not in scopeInstance.resolved[target.id].resolved
+        ):
+            continue
+
+        if component.resolved[target.id].enabled:
+            fillcolor = "lightgrey" if component.type == model.Kind.LIB else "lightblue"
+            shape = "plaintext" if not scope == component.id else "box"
+
+            g.node(
+                component.id,
+                f"<<B>{component.id}</B><BR/>{vt100.wordwrap(component.decription, 40,newline='<BR/>')}>",
+                shape=shape,
+                style="filled",
+                fillcolor=fillcolor,
+            )
+
+            for req in component.requires:
+                g.edge(component.id, req)
+
+            for req in component.provides:
+                isChosen = target.routing.get(req, None) == component.id
+
+                g.edge(
+                    req,
+                    component.id,
+                    arrowhead="none",
+                    color=("blue" if isChosen else "black"),
+                )
+        elif showDisabled:
+            g.node(
+                component.id,
+                f"<<B>{component.id}</B><BR/>{vt100.wordwrap(component.decription, 40,newline='<BR/>')}<BR/><BR/><I>{vt100.wordwrap(str(component.resolved[target.id].reason), 40,newline='<BR/>')}</I>>",
+                shape="plaintext",
+                style="filled",
+                fontcolor="#999999",
+                fillcolor="#eeeeee",
+            )
+
+            for req in component.requires:
+                g.edge(component.id, req, color="#aaaaaa")
+
+            for req in component.provides:
+                g.edge(req, component.id, arrowhead="none", color="#aaaaaa")
+
+    g.view(filename=os.path.join(target.builddir, "graph.gv"))
+
+
+@cli.command("g", "graph", "Show the dependency graph")
+def graphCmd(args: cli.Args):
+    registry = model.Registry.use(args)
+    target = model.Target.use(args)
+
+    scope = cast(Optional[str], args.tryConsumeOpt("scope"))
+    onlyLibs = args.consumeOpt("only-libs", False) is True
+    showDisabled = args.consumeOpt("show-disabled", False) is True
+
+    view(registry, target, scope=scope, showExe=not onlyLibs, showDisabled=showDisabled)

cutekit/jexpr.py

@@ -0,0 +1,80 @@
+import os
+import json
+import re
+import tomllib
+
+from pathlib import Path
+
+from typing import Any, Optional, cast, Callable, Final
+from . import shell
+
+Json = Any
+Builtin = Callable[..., Json]
+
+BUILTINS: Final[dict[str, Builtin]] = {
+    "uname": lambda arg, ctx: getattr(shell.uname(), arg).lower(),
+    "include": lambda arg, ctx: evalRead(Path(arg)),
+    "evalRead": lambda arg, ctx: evalRead(Path(arg)),
+    "join": lambda lhs, rhs, ctx: cast(
+        Json, {**lhs, **rhs} if isinstance(lhs, dict) else lhs + rhs
+    ),
+    "concat": lambda *args, ctx: "".join(args),
+    "first": lambda arg, ctx: arg[0],
+    "last": lambda arg, ctx: arg[-1],
+    "eval": lambda arg, ctx: eval(arg, ctx["filepath"]),
+    "read": lambda arg, ctx: read(arg),
+    "exec": lambda *args, ctx: shell.popen(*args).splitlines(),
+    "latest": lambda arg, ctx: shell.latest(arg),
+    "abspath": lambda *args, ctx: os.path.normpath(
+        os.path.join(os.path.dirname(ctx["filepath"]), *args)
+    ),
+}
+
+
+def eval(jexpr: Json, filePath: Path) -> Json:
+    if isinstance(jexpr, dict):
+        result = {}
+        for k in cast(dict[str, Json], jexpr):
+            result[k] = eval(jexpr[k], filePath)
+        return cast(Json, result)
+    elif isinstance(jexpr, list):
+        jexpr = cast(list[Json], jexpr)
+        if len(jexpr) > 0 and isinstance(jexpr[0], str) and jexpr[0].startswith("@"):
+            funcName = jexpr[0][1:]
+            if funcName in BUILTINS:
+                return BUILTINS[funcName](
+                    *eval(jexpr[1:], filePath), ctx={"filepath": filePath}
+                )
+
+            raise RuntimeError(f"Unknown macro {funcName}")
+        else:
+            return list(map(lambda j: eval(j, filePath), jexpr))
+    else:
+        return jexpr
+
+
+def extraSchema(toml: str) -> Optional[str]:
+    schemaRegex = re.compile(r"#:schema\s+(.*)")
+    schema = schemaRegex.search(toml)
+    return schema.group(1) if schema else None
+
+
+def read(path: Path) -> Json:
+    try:
+        with open(path, "r") as f:
+            if path.suffix == ".toml":
+                tomlStr = f.read()
+                toml = tomllib.loads(tomlStr)
+                schema = extraSchema(tomlStr)
+                if schema:
+                    toml["$schema"] = schema
+                return toml
+            else:
+                return json.load(f)
+    except Exception as e:
+        raise RuntimeError(f"Failed to read {path}: {e}")
+
+
+def evalRead(path: Path) -> Json:
+    data = read(path)
+    return eval(data, path)

cutekit/mixins.py

@@ -0,0 +1,83 @@
+from typing import Callable
+
+from . import model
+
+Mixin = Callable[[model.Target, model.Tools], model.Tools]
+
+
+def patchToolArgs(tools: model.Tools, toolSpec: str, args: list[str]):
+    tools[toolSpec].args += args
+
+
+def prefixToolCmd(tools: model.Tools, toolSpec: str, prefix: str):
+    tools[toolSpec].cmd = prefix + " " + tools[toolSpec].cmd
+
+
+def mixinCache(target: model.Target, tools: model.Tools) -> model.Tools:
+    prefixToolCmd(tools, "cc", "ccache")
+    prefixToolCmd(tools, "cxx", "ccache")
+    return tools
+
+
+def makeMixinSan(san: str) -> Mixin:
+    def mixinSan(target: model.Target, tools: model.Tools) -> model.Tools:
+        patchToolArgs(tools, "cc", [f"-fsanitize={san}"])
+        patchToolArgs(tools, "cxx", [f"-fsanitize={san}"])
+        patchToolArgs(tools, "ld", [f"-fsanitize={san}"])
+
+        return tools
+
+    return mixinSan
+
+
+def makeMixinOptimize(level: str) -> Mixin:
+    def mixinOptimize(target: model.Target, tools: model.Tools) -> model.Tools:
+        patchToolArgs(tools, "cc", [f"-O{level}"])
+        patchToolArgs(tools, "cxx", [f"-O{level}"])
+
+        return tools
+
+    return mixinOptimize
+
+
+def mixinDebug(target: model.Target, tools: model.Tools) -> model.Tools:
+    patchToolArgs(tools, "cc", ["-g", "-gdwarf-4"])
+    patchToolArgs(tools, "cxx", ["-g", "-gdwarf-4"])
+
+    return tools
+
+
+def makeMixinTune(tune: str) -> Mixin:
+    def mixinTune(target: model.Target, tools: model.Tools) -> model.Tools:
+        patchToolArgs(tools, "cc", [f"-mtune={tune}"])
+        patchToolArgs(tools, "cxx", [f"-mtune={tune}"])
+
+        return tools
+
+    return mixinTune
+
+
+mixins: dict[str, Mixin] = {
+    "cache": mixinCache,
+    "debug": mixinDebug,
+    "asan": makeMixinSan("address"),
+    "msan": makeMixinSan("memory"),
+    "tsan": makeMixinSan("thread"),
+    "ubsan": makeMixinSan("undefined"),
+    "tune": makeMixinTune("native"),
+    "fast": makeMixinOptimize("fast"),
+    "o3": makeMixinOptimize("3"),
+    "o2": makeMixinOptimize("2"),
+    "o1": makeMixinOptimize("1"),
+    "o0": makeMixinOptimize("0"),
+}
+
+
+def append(mixinSpec: str, mixin: Mixin):
+    mixins[mixinSpec] = mixin
+
+
+def byId(id: str) -> Mixin:
+    if id not in mixins:
+        raise RuntimeError(f"Unknown mixin {id}")
+    return mixins[id]

cutekit/model.py

@@ -0,0 +1,632 @@
+import os
+import logging
+import dataclasses as dt
+
+
+from enum import Enum
+from typing import Any, Generator, Optional, Type, cast
+from pathlib import Path
+from dataclasses_json import DataClassJsonMixin
+
+from cutekit import const, shell
+
+from . import jexpr, compat, utils, cli, vt100
+
+_logger = logging.getLogger(__name__)
+
+Props = dict[str, Any]
+
+
+class Kind(Enum):
+    UNKNOWN = "unknown"
+    PROJECT = "project"
+    TARGET = "target"
+    LIB = "lib"
+    EXE = "exe"
+
+
+# --- Manifest --------------------------------------------------------------- #
+
+
+@dt.dataclass
+class Manifest(DataClassJsonMixin):
+    id: str
+    type: Kind = dt.field(default=Kind.UNKNOWN)
+    path: str = dt.field(default="")
+    SUFFIXES = [".json", ".toml"]
+    SUFFIXES_GLOBS = ["*.json", "*.toml"]
+
+    @staticmethod
+    def parse(path: Path, data: dict[str, Any]) -> "Manifest":
+        """
+        Parse a manifest from a given path and data
+        """
+        compat.ensureSupportedManifest(data, path)
+        kind = Kind(data["type"])
+        del data["$schema"]
+        obj = KINDS[kind].from_dict(data)
+        obj.path = str(path)
+        return obj
+
+    @staticmethod
+    def tryLoad(path: Path) -> Optional["Manifest"]:
+        for suffix in Manifest.SUFFIXES:
+            pathWithSuffix = path.with_suffix(suffix)
+            if pathWithSuffix.exists():
+                _logger.debug(f"Loading manifest from '{pathWithSuffix}'")
+                return Manifest.parse(pathWithSuffix, jexpr.evalRead(pathWithSuffix))
+        return None
+
+    @staticmethod
+    def load(path: Path) -> "Manifest":
+        """
+        Load a manifest from a given path
+        """
+        manifest = Manifest.tryLoad(path)
+        if manifest is None:
+            raise RuntimeError(f"Could not find manifest at '{path}'")
+        return manifest
+
+    def dirname(self) -> str:
+        """
+        Return the directory of the manifest
+        """
+        return os.path.relpath(os.path.dirname(self.path), Path.cwd())
+
+    def subpath(self, path) -> Path:
+        return Path(self.dirname()) / path
+
+    def ensureType(self, t: Type[utils.T]) -> utils.T:
+        """
+        Ensure that the manifest is of a given type
+        """
+        if not isinstance(self, t):
+            raise RuntimeError(
+                f"{self.path} should be a {type.__name__} manifest but is a {self.__class__.__name__} manifest"
+            )
+        return cast(utils.T, self)
+
+
+# --- Project ---------------------------------------------------------------- #
+
+_project: Optional["Project"] = None
+
+
+@dt.dataclass
+class Extern(DataClassJsonMixin):
+    git: str
+    tag: str
+
+
+@dt.dataclass
+class Project(Manifest):
+    description: str = dt.field(default="(No description)")
+    extern: dict[str, Extern] = dt.field(default_factory=dict)
+
+    @property
+    def externDirs(self) -> list[str]:
+        res = map(lambda e: os.path.join(const.EXTERN_DIR, e), self.extern.keys())
+        return list(res)
+
+    @staticmethod
+    def topmost() -> Optional["Project"]:
+        cwd = Path.cwd()
+        topmost: Optional["Project"] = None
+        while str(cwd) != cwd.root:
+            projectManifest = Manifest.tryLoad(cwd / "project")
+            if projectManifest is not None:
+                topmost = projectManifest.ensureType(Project)
+            cwd = cwd.parent
+        return topmost
+
+    @staticmethod
+    def ensure() -> "Project":
+        """
+        Ensure that a project exists in the current directory or any parent directory
+        and chdir to the root of the project.
+        """
+        project = Project.topmost()
+        if project is None:
+            raise RuntimeError(
+                "No project found in this directory or any parent directory"
+            )
+        os.chdir(project.dirname())
+        return project
+
+    @staticmethod
+    def at(path: Path) -> Optional["Project"]:
+        projectManifest = Manifest.tryLoad(path / "project")
+        if projectManifest is None:
+            return None
+        return projectManifest.ensureType(Project)
+
+    @staticmethod
+    def fetchs(extern: dict[str, Extern]):
+        for extSpec, ext in extern.items():
+            extPath = os.path.join(const.EXTERN_DIR, extSpec)
+
+            if os.path.exists(extPath):
+                print(f"Skipping {extSpec}, already installed")
+                continue
+
+            print(f"Installing {extSpec}-{ext.tag} from {ext.git}...")
+            shell.popen(
+                "git",
+                "clone",
+                "--quiet",
+                "--depth",
+                "1",
+                "--branch",
+                ext.tag,
+                ext.git,
+                extPath,
+            )
+            project = Project.at(Path(extPath))
+            if project is not None:
+                Project.fetchs(project.extern)
+
+    @staticmethod
+    def use(args: cli.Args) -> "Project":
+        global _project
+        if _project is None:
+            _project = Project.ensure()
+        return _project
+
+
+@cli.command("i", "install", "Install required external packages")
+def installCmd(args: cli.Args):
+    project = Project.use(args)
+    Project.fetchs(project.extern)
+
+
+@cli.command("I", "init", "Initialize a new project")
+def initCmd(args: cli.Args):
+    import requests
+
+    repo = args.consumeOpt("repo", const.DEFAULT_REPO_TEMPLATES)
+    list = args.consumeOpt("list")
+
+    template = args.consumeArg()
+    name = args.consumeArg()
+
+    _logger.info("Fetching registry...")
+
+    r = requests.get(f"https://raw.githubusercontent.com/{repo}/main/registry.json")
+
+    if r.status_code != 200:
+        _logger.error("Failed to fetch registry")
+        exit(1)
+
+    registry = r.json()
+
+    if list:
+        print(
+            "\n".join(f"* {entry['id']} - {entry['description']}" for entry in registry)
+        )
+        return
+
+    if not template:
+        raise RuntimeError("Template not specified")
+
+    def template_match(t: jexpr.Json) -> str:
+        return t["id"] == template
+
+    if not any(filter(template_match, registry)):
+        raise LookupError(f"Couldn't find a template named {template}")
+
+    if not name:
+        _logger.info(f"No name was provided, defaulting to {template}")
+        name = template
+
+    if os.path.exists(name):
+        raise RuntimeError(f"Directory {name} already exists")
+
+    print(f"Creating project {name} from template {template}...")
+    shell.cloneDir(f"https://github.com/{repo}", template, name)
+    print(f"Project {name} created\n")
+
+    print("We suggest that you begin by typing:")
+    print(f"  {vt100.GREEN}cd {name}{vt100.RESET}")
+    print(
+        f"  {vt100.GREEN}cutekit install{vt100.BRIGHT_BLACK} # Install external packages{vt100.RESET}"
+    )
+    print(
+        f"  {vt100.GREEN}cutekit build{vt100.BRIGHT_BLACK}  # Build the project{vt100.RESET}"
+    )
+
+
+# --- Target ----------------------------------------------------------------- #
+
+
+@dt.dataclass
+class Tool(DataClassJsonMixin):
+    cmd: str = dt.field(default="")
+    args: list[str] = dt.field(default_factory=list)
+    files: list[str] = dt.field(default_factory=list)
+
+
+Tools = dict[str, Tool]
+
+DEFAULT_TOOLS: Tools = {
+    "cp": Tool("cp"),
+}
+
+
+@dt.dataclass
+class Target(Manifest):
+    props: Props = dt.field(default_factory=dict)
+    tools: Tools = dt.field(default_factory=dict)
+    routing: dict[str, str] = dt.field(default_factory=dict)
+
+    @property
+    def hashid(self) -> str:
+        return utils.hash((self.props, [v.to_dict() for k, v in self.tools.items()]))
+
+    @property
+    def builddir(self) -> str:
+        return os.path.join(const.BUILD_DIR, f"{self.id}-{self.hashid[:8]}")
+
+    @staticmethod
+    def use(args: cli.Args) -> "Target":
+        registry = Registry.use(args)
+        targetSpec = str(args.consumeOpt("target", "host-" + shell.uname().machine))
+        return registry.ensure(targetSpec, Target)
+
+    def route(self, componentSpec: str):
+        """
+        Route a component spec to a target specific component spec
+        """
+        return (
+            self.routing[componentSpec]
+            if componentSpec in self.routing
+            else componentSpec
+        )
+
+
+# --- Component -------------------------------------------------------------- #
+
+
+@dt.dataclass
+class Resolved:
+    reason: Optional[str] = None
+    resolved: list[str] = dt.field(default_factory=list)
+
+    @property
+    def enabled(self) -> bool:
+        return self.reason is None
+
+
+@dt.dataclass
+class Component(Manifest):
+    decription: str = dt.field(default="(No description)")
+    props: Props = dt.field(default_factory=dict)
+    tools: Tools = dt.field(default_factory=dict)
+    enableIf: dict[str, list[Any]] = dt.field(default_factory=dict)
+    requires: list[str] = dt.field(default_factory=list)
+    provides: list[str] = dt.field(default_factory=list)
+    subdirs: list[str] = dt.field(default_factory=list)
+    injects: list[str] = dt.field(default_factory=list)
+    resolved: dict[str, Resolved] = dt.field(default_factory=dict)
+
+    def isEnabled(self, target: Target) -> tuple[bool, str]:
+        for k, v in self.enableIf.items():
+            if k not in target.props:
+                _logger.info(f"Component {self.id} disabled by missing {k} in target")
+                return False, f"Missing props '{k}' in target"
+
+            if target.props[k] not in v:
+                vStrs = [f"'{str(x)}'" for x in v]
+                _logger.info(
+                    f"Component {self.id} disabled by {k}={target.props[k]} not in {v}"
+                )
+                return (
+                    False,
+                    f"Props missmatch for '{k}': Got '{target.props[k]}' but expected {', '.join(vStrs)}",
+                )
+
+        return True, ""
+
+
+KINDS: dict[Kind, Type[Manifest]] = {
+    Kind.PROJECT: Project,
+    Kind.TARGET: Target,
+    Kind.LIB: Component,
+    Kind.EXE: Component,
+}
+
+# --- Dependency resolution -------------------------------------------------- #
+
+
+@dt.dataclass
+class Resolver:
+    _registry: "Registry"
+    _target: Target
+    _mappings: dict[str, list[Component]] = dt.field(default_factory=dict)
+    _cache: dict[str, Resolved] = dt.field(default_factory=dict)
+    _baked = False
+
+    def _bake(self):
+        """
+        Bake the resolver by building a mapping of all
+        components that provide a given spec.
+        """
+
+        if self._baked:
+            return
+
+        for c in self._registry.iter(Component):
+            for p in c.provides + [c.id]:
+                if p not in self._mappings and [0]:
+                    self._mappings[p] = []
+                self._mappings[p].append(c)
+
+        # Overide with target routing since it has priority
+        # over component provides and id
+        for k, v in self._target.routing.items():
+            component = self._registry.lookup(v, Component)
+            self._mappings[k] = [component] if component else []
+
+        self._baked = True
+
+    def _provider(self, spec: str) -> tuple[Optional[str], str]:
+        """
+        Returns the provider for a given spec.
+        """
+        result = self._mappings.get(spec, [])
+
+        if len(result) == 1:
+            enabled, reason = result[0].isEnabled(self._target)
+            if not enabled:
+                return (None, reason)
+
+        def checkIsEnabled(c: Component) -> bool:
+            enabled, reason = c.isEnabled(self._target)
+            if not enabled:
+                _logger.info(f"Component {c.id} cannot provide '{spec}': {reason}")
+            return enabled
+
+        result = list(filter(checkIsEnabled, result))
+
+        if result == []:
+            return (None, f"No provider for '{spec}'")
+
+        if len(result) > 1:
+            ids = list(map(lambda x: x.id, result))
+            return (None, f"Multiple providers for '{spec}': {','.join(ids)}")
+
+        return (result[0].id, "")
+
+    def resolve(self, what: str, stack: list[str] = []) -> Resolved:
+        """
+        Resolve a given spec to a list of components.
+        """
+        self._bake()
+
+        if what in self._cache:
+            return self._cache[what]
+
+        keep, unresolvedReason = self._provider(what)
+
+        if not keep:
+            _logger.error(f"Dependency '{what}' not found: {unresolvedReason}")
+            self._cache[what] = Resolved(reason=unresolvedReason)
+            return self._cache[what]
+
+        if keep in self._cache:
+            return self._cache[keep]
+
+        if keep in stack:
+            raise RuntimeError(
+                f"Dependency loop while resolving '{what}': {stack} -> {keep}"
+            )
+
+        stack.append(keep)
+
+        component = self._registry.lookup(keep, Component)
+        if not component:
+            return Resolved(reason="No provider for 'myembed'")
+
+        result: list[str] = []
+
+        for req in component.requires:
+            reqResolved = self.resolve(req, stack)
+            if reqResolved.reason:
+                stack.pop()
+
+                self._cache[keep] = Resolved(reason=reqResolved.reason)
+                return self._cache[keep]
+
+            result.extend(reqResolved.resolved)
+
+        stack.pop()
+        result.insert(0, keep)
+        self._cache[keep] = Resolved(resolved=utils.uniq(result))
+        return self._cache[keep]
+
+
+# --- Registry --------------------------------------------------------------- #
+
+_registry: Optional["Registry"] = None
+
+
+@dt.dataclass
+class Registry(DataClassJsonMixin):
+    project: Project
+    manifests: dict[str, Manifest] = dt.field(default_factory=dict)
+
+    def _append(self, m: Optional[Manifest]) -> Optional[Manifest]:
+        """
+        Append a manifest to the model
+        """
+        if m is None:
+            return m
+
+        if m.id in self.manifests:
+            raise RuntimeError(
+                f"Duplicated manifest '{m.id}' at '{m.path}' already loaded from '{self.manifests[m.id].path}'"
+            )
+
+        self.manifests[m.id] = m
+        return m
+
+    def iter(self, type: Type[utils.T]) -> Generator[utils.T, None, None]:
+        """
+        Iterate over all manifests of a given type
+        """
+
+        for m in self.manifests.values():
+            if isinstance(m, type):
+                yield m
+
+    def iterEnabled(self, target: Target) -> Generator[Component, None, None]:
+        for c in self.iter(Component):
+            resolve = c.resolved[target.id]
+            if resolve.enabled:
+                yield c
+
+    def lookup(
+        self, name: str, type: Type[utils.T], includeProvides: bool = False
+    ) -> Optional[utils.T]:
+        """
+        Lookup a manifest of a given type by name
+        """
+
+        if name in self.manifests:
+            m = self.manifests[name]
+            if isinstance(m, type):
+                return m
+
+        if includeProvides and type is Component:
+            for m in self.iter(Component):
+                if name in m.provides:
+                    return m  # type: ignore
+
+        return None
+
+    def ensure(self, name: str, type: Type[utils.T]) -> utils.T:
+        """
+        Ensure that a manifest of a given type exists
+        and return it.
+        """
+
+        m = self.lookup(name, type)
+        if not m:
+            raise RuntimeError(f"Could not find {type.__name__} '{name}'")
+        return m
+
+    @staticmethod
+    def use(args: cli.Args) -> "Registry":
+        global _registry
+
+        if _registry is not None:
+            return _registry
+
+        project = Project.use(args)
+        mixins = str(args.consumeOpt("mixins", "")).split(",")
+        if mixins == [""]:
+            mixins = []
+        props = cast(dict[str, str], args.consumePrefix("prop:"))
+
+        _registry = Registry.load(project, mixins, props)
+        return _registry
+
+    @staticmethod
+    def load(project: Project, mixins: list[str], props: Props) -> "Registry":
+        r = Registry(project)
+        r._append(project)
+
+        # Lookup and load all extern projects
+        for externDir in project.externDirs:
+            extern = r._append(
+                Manifest.tryLoad(Path(externDir) / "project")
+                or Manifest.tryLoad(Path(externDir) / "manifest")
+            )
+
+            if extern is not None:
+                _logger.warn("Extern project does not have a project or manifest")
+
+        # Load all manifests from projects
+        for project in list(r.iter(Project)):
+            targetDir = os.path.join(project.dirname(), const.TARGETS_DIR)
+            targetFiles = shell.find(targetDir, Manifest.SUFFIXES_GLOBS)
+
+            for targetFile in targetFiles:
+                r._append(Manifest.load(Path(targetFile)).ensureType(Target))
+
+            componentFiles = shell.find(
+                os.path.join(project.dirname(), const.SRC_DIR),
+                ["manifest" + s for s in Manifest.SUFFIXES],
+            )
+
+            rootComponent = Manifest.tryLoad(Path(project.dirname()) / "manifest")
+            if rootComponent is not None:
+                r._append(rootComponent)
+
+            for componentFile in componentFiles:
+                r._append(Manifest.load(Path(componentFile)).ensureType(Component))
+
+        # Resolve all dependencies for all targets
+        for target in r.iter(Target):
+            target.props |= props
+            resolver = Resolver(r, target)
+
+            # Apply injects
+            for c in r.iter(Component):
+                if c.isEnabled(target)[0]:
+                    for inject in c.injects:
+                        victim = r.lookup(inject, Component)
+                        if not victim:
+                            raise RuntimeError(f"Cannot find component '{inject}'")
+                        victim.requires += [c.id]
+
+            # Resolve all components
+            for c in r.iter(Component):
+                resolved = resolver.resolve(c.id)
+                if resolved.reason:
+                    _logger.info(f"Component '{c.id}' disabled: {resolved.reason}")
+                c.resolved[target.id] = resolved
+
+            # Resolve tooling
+            tools: Tools = target.tools
+
+            # Merge in default tools
+            for k, v in DEFAULT_TOOLS.items():
+                if k not in tools:
+                    tools[k] = dt.replace(v)
+
+            from . import mixins as mxs
+
+            for mix in mixins:
+                mixin = mxs.byId(mix)
+                tools = mixin(target, tools)
+
+            # Apply tooling from components
+            for c in r.iter(Component):
+                if c.resolved[target.id].enabled:
+                    for k, v in c.tools.items():
+                        tools[k].args += v.args
+
+        return r
+
+
+@cli.command("l", "list", "List all components and targets")
+def listCmd(args: cli.Args):
+    registry = Registry.use(args)
+
+    components = list(registry.iter(Component))
+    targets = list(registry.iter(Target))
+
+    vt100.title("Components")
+    if len(components) == 0:
+        print(vt100.p("(No components available)"))
+    else:
+        print(vt100.p(", ".join(map(lambda m: m.id, components))))
+    print()
+
+    vt100.title("Targets")
+
+    if len(targets) == 0:
+        print(vt100.p("(No targets available)"))
+    else:
+        print(vt100.p(", ".join(map(lambda m: m.id, targets))))
+    print()

cutekit/ninja.py

@@ -23,13 +23,13 @@ use Python.
 """
 
 import textwrap
-from typing import TextIO, Union
 
-from osdk.utils import asList
+from typing import TextIO, Union
+from . import utils
 
 
 def escapePath(word: str) -> str:
-    return word.replace('$ ', '$$ ').replace(' ', '$ ').replace(':', '$:')
+    return word.replace("$ ", "$$ ").replace(" ", "$ ").replace(":", "$:")
 
 
 VarValue = Union[int, str, list[str], None]
@@ -42,92 +42,98 @@ class Writer(object):
         self.width = width
 
     def newline(self) -> None:
-        self.output.write('\n')
+        self.output.write("\n")
 
     def comment(self, text: str) -> None:
-        for line in textwrap.wrap(text, self.width - 2, break_long_words=False,
-                                  break_on_hyphens=False):
-            self.output.write('# ' + line + '\n')
+        for line in textwrap.wrap(
+            text, self.width - 2, break_long_words=False, break_on_hyphens=False
+        ):
+            self.output.write("# " + line + "\n")
 
-    def separator(self, text : str) -> None:
-        self.output.write(f"# --- {text} ---" + '-' *
-                          (self.width - 10 - len(text)) + " #\n\n")
+    def separator(self, text: str) -> None:
+        self.output.write(
+            f"# --- {text} ---" + "-" * (self.width - 10 - len(text)) + " #\n\n"
+        )
 
     def variable(self, key: str, value: VarValue, indent: int = 0) -> None:
         if value is None:
             return
         if isinstance(value, list):
-            value = ' '.join(filter(None, value))  # Filter out empty strings.
-        self._line('%s = %s' % (key, value), indent)
+            value = " ".join(filter(None, value))  # Filter out empty strings.
+        self._line("%s = %s" % (key, value), indent)
 
     def pool(self, name: str, depth: int) -> None:
-        self._line('pool %s' % name)
-        self.variable('depth', depth, indent=1)
+        self._line("pool %s" % name)
+        self.variable("depth", depth, indent=1)
 
-    def rule(self,
-             name: str,
-             command: VarValue,
-             description: Union[str, None] = None,
-             depfile: VarValue = None,
-             generator: VarValue = False,
-             pool: VarValue = None,
-             restat: bool = False,
-             rspfile: VarValue = None,
-             rspfile_content: VarValue = None,
-             deps: VarValue = None) -> None:
-        self._line('rule %s' % name)
-        self.variable('command', command, indent=1)
+    def rule(
+        self,
+        name: str,
+        command: VarValue,
+        description: Union[str, None] = None,
+        depfile: VarValue = None,
+        generator: VarValue = False,
+        pool: VarValue = None,
+        restat: bool = False,
+        rspfile: VarValue = None,
+        rspfile_content: VarValue = None,
+        deps: VarValue = None,
+    ) -> None:
+        self._line("rule %s" % name)
+        self.variable("command", command, indent=1)
         if description:
-            self.variable('description', description, indent=1)
+            self.variable("description", description, indent=1)
         if depfile:
-            self.variable('depfile', depfile, indent=1)
+            self.variable("depfile", depfile, indent=1)
         if generator:
-            self.variable('generator', '1', indent=1)
+            self.variable("generator", "1", indent=1)
         if pool:
-            self.variable('pool', pool, indent=1)
+            self.variable("pool", pool, indent=1)
         if restat:
-            self.variable('restat', '1', indent=1)
+            self.variable("restat", "1", indent=1)
         if rspfile:
-            self.variable('rspfile', rspfile, indent=1)
+            self.variable("rspfile", rspfile, indent=1)
         if rspfile_content:
-            self.variable('rspfile_content', rspfile_content, indent=1)
+            self.variable("rspfile_content", rspfile_content, indent=1)
         if deps:
-            self.variable('deps', deps, indent=1)
+            self.variable("deps", deps, indent=1)
 
-    def build(self,
-              outputs: Union[str, list[str]],
-              rule: str,
-              inputs: Union[VarPath, None],
-              implicit: VarPath = None,
-              order_only: VarPath = None,
-              variables: Union[dict[str, str], None] = None,
-              implicit_outputs: VarPath = None,
-              pool: Union[str, None] = None,
-              dyndep: Union[str, None] = None) -> list[str]:
-        outputs = asList(outputs)
+    def build(
+        self,
+        outputs: Union[str, list[str]],
+        rule: str,
+        inputs: Union[VarPath, None],
+        implicit: VarPath = None,
+        order_only: VarPath = None,
+        variables: Union[dict[str, str], None] = None,
+        implicit_outputs: VarPath = None,
+        pool: Union[str, None] = None,
+        dyndep: Union[str, None] = None,
+    ) -> list[str]:
+        outputs = utils.asList(outputs)
         out_outputs = [escapePath(x) for x in outputs]
-        all_inputs = [escapePath(x) for x in asList(inputs)]
+        all_inputs = [escapePath(x) for x in utils.asList(inputs)]
 
         if implicit:
-            implicit = [escapePath(x) for x in asList(implicit)]
-            all_inputs.append('|')
+            implicit = [escapePath(x) for x in utils.asList(implicit)]
+            all_inputs.append("|")
             all_inputs.extend(implicit)
         if order_only:
-            order_only = [escapePath(x) for x in asList(order_only)]
-            all_inputs.append('||')
+            order_only = [escapePath(x) for x in utils.asList(order_only)]
+            all_inputs.append("||")
             all_inputs.extend(order_only)
         if implicit_outputs:
-            implicit_outputs = [escapePath(x)
-                                for x in asList(implicit_outputs)]
-            out_outputs.append('|')
+            implicit_outputs = [escapePath(x) for x in utils.asList(implicit_outputs)]
+            out_outputs.append("|")
             out_outputs.extend(implicit_outputs)
 
-        self._line('build %s: %s' % (' '.join(out_outputs),
-                                     ' '.join([rule] + all_inputs)))
+        self._line(
+            "build %s: %s" % (" ".join(out_outputs), " ".join([rule] + all_inputs))
+        )
         if pool is not None:
-            self._line('  pool = %s' % pool)
+            self._line("  pool = %s" % pool)
         if dyndep is not None:
-            self._line('  dyndep = %s' % dyndep)
+            self._line("  dyndep = %s" % dyndep)
 
         if variables:
             iterator = iter(variables.items())
@@ -138,58 +144,59 @@ class Writer(object):
         return outputs
 
     def include(self, path: str) -> None:
-        self._line('include %s' % path)
+        self._line("include %s" % path)
 
     def subninja(self, path: str) -> None:
-        self._line('subninja %s' % path)
+        self._line("subninja %s" % path)
 
     def default(self, paths: VarPath) -> None:
-        self._line('default %s' % ' '.join(asList(paths)))
+        self._line("default %s" % " ".join(utils.asList(paths)))
 
     def _count_dollars_before_index(self, s: str, i: int) -> int:
         """Returns the number of '$' characters right in front of s[i]."""
         dollar_count = 0
         dollar_index = i - 1
-        while dollar_index > 0 and s[dollar_index] == '$':
+        while dollar_index > 0 and s[dollar_index] == "$":
             dollar_count += 1
             dollar_index -= 1
         return dollar_count
 
     def _line(self, text: str, indent: int = 0) -> None:
         """Write 'text' word-wrapped at self.width characters."""
-        leading_space = '  ' * indent
+        leading_space = "  " * indent
         while len(leading_space) + len(text) > self.width:
             # The text is too wide; wrap if possible.
 
             # Find the rightmost space that would obey our width constraint and
             # that's not an escaped space.
-            available_space = self.width - len(leading_space) - len(' $')
+            available_space = self.width - len(leading_space) - len(" $")
             space = available_space
             while True:
-                space = text.rfind(' ', 0, space)
-                if (space < 0 or
-                        self._count_dollars_before_index(text, space) % 2 == 0):
+                space = text.rfind(" ", 0, space)
+                if space < 0 or self._count_dollars_before_index(text, space) % 2 == 0:
                     break
 
             if space < 0:
                 # No such space; just use the first unescaped space we can find.
                 space = available_space - 1
                 while True:
-                    space = text.find(' ', space + 1)
-                    if (space < 0 or
-                            self._count_dollars_before_index(text, space) % 2 == 0):
+                    space = text.find(" ", space + 1)
+                    if (
+                        space < 0
+                        or self._count_dollars_before_index(text, space) % 2 == 0
+                    ):
                         break
             if space < 0:
                 # Give up on breaking.
                 break
 
-            self.output.write(leading_space + text[0:space] + ' $\n')
-            text = text[space+1:]
+            self.output.write(leading_space + text[0:space] + " $\n")
+            text = text[space + 1 :]
 
             # Subsequent lines are continuations, so indent them.
-            leading_space = '  ' * (indent+2)
+            leading_space = "  " * (indent + 2)
 
-        self.output.write(leading_space + text + '\n')
+        self.output.write(leading_space + text + "\n")
 
     def close(self) -> None:
         self.output.close()
@@ -198,6 +205,6 @@ class Writer(object):
 def escape(string: str) -> str:
     """Escape a string such that it can be embedded into a Ninja file without
     further interpretation."""
-    assert '\n' not in string, 'Ninja syntax does not allow newlines'
+    assert "\n" not in string, "Ninja syntax does not allow newlines"
     # We only have one special metacharacter: '$'.
-    return string.replace('$', '$$')
+    return string.replace("$", "$$")

cutekit/plugins.py

@@ -0,0 +1,43 @@
+import os
+import logging
+
+from . import shell, model, const
+
+import importlib.util as importlib
+
+_logger = logging.getLogger(__name__)
+
+
+def load(path: str):
+    _logger.info(f"Loading plugin {path}")
+    spec = importlib.spec_from_file_location("plugin", path)
+
+    if not spec or not spec.loader:
+        _logger.error(f"Failed to load plugin {path}")
+        return None
+
+    module = importlib.module_from_spec(spec)
+    spec.loader.exec_module(module)
+
+
+def loadAll():
+    _logger.info("Loading plugins...")
+
+    project = model.Project.topmost()
+    if project is None:
+        _logger.info("Not in project, skipping plugin loading")
+        return
+    paths = list(
+        map(lambda e: os.path.join(const.EXTERN_DIR, e), project.extern.keys())
+    ) + ["."]
+
+    for dirname in paths:
+        pluginDir = os.path.join(project.dirname(), dirname, const.META_DIR, "plugins")
+
+        for files in shell.readdir(pluginDir):
+            if files.endswith(".py"):
+                plugin = load(os.path.join(pluginDir, files))
+
+                if plugin:
+                    _logger.info(f"Loaded plugin {plugin.name}")
+                    plugin.init()

cutekit/rules.py

@@ -0,0 +1,67 @@
+from typing import Optional
+
+
+class Rule:
+    id: str
+    fileIn: list[str]
+    fileOut: list[str]
+    rule: str
+    args: list[str]
+    deps: Optional[str] = None
+
+    def __init__(
+        self,
+        id: str,
+        fileIn: list[str],
+        fileOut: list[str],
+        rule: str,
+        args: list[str] = [],
+        deps: Optional[str] = None,
+    ):
+        self.id = id
+        self.fileIn = fileIn
+        self.fileOut = fileOut
+        self.rule = rule
+        self.args = args
+        self.deps = deps
+
+
+rules: dict[str, Rule] = {
+    "cp": Rule("cp", ["*"], ["*"], "$in $out"),
+    "cc": Rule(
+        "cc",
+        ["*.c"],
+        ["*.o"],
+        "-c -o $out $in -MD -MF $out.d $flags $cincs $cdefs",
+        ["-std=gnu2x", "-Wall", "-Wextra", "-Werror"],
+        "$out.d",
+    ),
+    "cxx": Rule(
+        "cxx",
+        ["*.cpp", "*.cc", "*.cxx"],
+        ["*.o"],
+        "-c -o $out $in -MD -MF $out.d $flags $cincs $cdefs",
+        ["-std=gnu++2b", "-Wall", "-Wextra", "-Werror", "-fno-exceptions", "-fno-rtti"],
+        "$out.d",
+    ),
+    "as": Rule("as", ["*.s", "*.asm", "*.S"], ["*.o"], "-o $out $in $flags"),
+    "ar": Rule("ar", ["*.o"], ["*.a"], "$flags $out $in"),
+    "ld": Rule("ld", ["*.o", "*.a"], ["*.out"], "-o $out $in $flags"),
+}
+
+
+def append(rule: Rule):
+    rules[rule.id] = rule
+
+
+def byFileIn(fileIn: str) -> Optional[Rule]:
+    for key in rules:
+        rule = rules[key]
+        for ext in rule.fileIn:
+            if fileIn.endswith(ext[1:]):
+                return rule
+    return None
+
+
+def byId(id: str) -> Optional[Rule]:
+    return rules.get(id, None)

cutekit/shell.py

@@ -8,15 +8,20 @@ import re
 import shutil
 import fnmatch
 import platform
+import logging
+import tempfile
 
-from osdk.logger import Logger
-from osdk import const
 
-logger = Logger("shell")
+from typing import Optional
+from . import const
+
+_logger = logging.getLogger(__name__)
 
 
 class Uname:
-    def __init__(self, sysname: str, nodename: str, release: str, version: str, machine: str):
+    def __init__(
+        self, sysname: str, nodename: str, release: str, version: str, machine: str
+    ):
         self.sysname = sysname
         self.nodename = nodename
         self.release = release
@@ -33,6 +38,8 @@ def uname() -> Uname:
             result.machine = "arm64"
         case "AMD64":
             result.machine = "x86_64"
+        case _:
+            pass
 
     return result
 
@@ -42,8 +49,10 @@ def sha256sum(path: str) -> str:
         return hashlib.sha256(f.read()).hexdigest()
 
 
-def find(path: str | list[str], wildcards: list[str] = [], recusive: bool = True) -> list[str]:
-    logger.log(f"Looking for files in {path} matching {wildcards}")
+def find(
+    path: str | list[str], wildcards: list[str] = [], recusive: bool = True
+) -> list[str]:
+    _logger.info(f"Looking for files in {path} matching {wildcards}")
 
     result: list[str] = []
 
@@ -79,7 +88,7 @@ def find(path: str | list[str], wildcards: list[str] = [], recusive: bool = True
 
 
 def mkdir(path: str) -> str:
-    logger.log(f"Creating directory {path}")
+    _logger.info(f"Creating directory {path}")
 
     try:
         os.makedirs(path)
@@ -90,7 +99,7 @@ def mkdir(path: str) -> str:
 
 
 def rmrf(path: str) -> bool:
-    logger.log(f"Removing directory {path}")
+    _logger.info(f"Removing directory {path}")
 
     if not os.path.exists(path):
         return False
@@ -98,22 +107,23 @@ def rmrf(path: str) -> bool:
     return True
 
 
-def wget(url: str, path: str | None = None) -> str:
+def wget(url: str, path: Optional[str] = None) -> str:
     import requests
 
     if path is None:
-        path = os.path.join(const.CACHE_DIR,
-            hashlib.sha256(url.encode('utf-8')).hexdigest())
+        path = os.path.join(
+            const.CACHE_DIR, hashlib.sha256(url.encode("utf-8")).hexdigest()
+        )
 
     if os.path.exists(path):
         return path
 
-    logger.log(f"Downloading {url} to {path}")
+    _logger.info(f"Downloading {url} to {path}")
 
     r = requests.get(url, stream=True)
     r.raise_for_status()
     mkdir(os.path.dirname(path))
-    with open(path, 'wb') as f:
+    with open(path, "wb") as f:
         for chunk in r.iter_content(chunk_size=8192):
             if chunk:
                 f.write(chunk)
@@ -121,48 +131,56 @@ def wget(url: str, path: str | None = None) -> str:
     return path
 
 
-def exec(*args: str):
-    logger.log(f"Executing {args}")
+def exec(*args: str, quiet: bool = False) -> bool:
+    _logger.info(f"Executing {args}")
 
     try:
-        proc = subprocess.run(args)
+        proc = subprocess.run(
+            args,
+            stdout=sys.stdout if not quiet else subprocess.PIPE,
+            stderr=sys.stderr if not quiet else subprocess.PIPE,
+        )
+
+        if proc.stdout:
+            _logger.info(proc.stdout.decode("utf-8"))
+
+        if proc.stderr:
+            _logger.error(proc.stderr.decode("utf-8"))
 
     except FileNotFoundError:
-        raise Exception(f"{args[0]}: Command not found")
+        raise RuntimeError(f"{args[0]}: Command not found")
 
     except KeyboardInterrupt:
-        raise Exception(f"{args[0]}: Interrupted")
+        raise RuntimeError(f"{args[0]}: Interrupted")
 
     if proc.returncode == -signal.SIGSEGV:
-        raise Exception(f"{args[0]}: Segmentation fault")
+        raise RuntimeError(f"{args[0]}: Segmentation fault")
 
     if proc.returncode != 0:
-        raise Exception(
-            f"{args[0]}: Process exited with code {proc.returncode}")
+        raise RuntimeError(f"{args[0]}: Process exited with code {proc.returncode}")
 
     return True
 
 
 def popen(*args: str) -> str:
-    logger.log(f"Executing {args}")
+    _logger.info(f"Executing {args}")
 
     try:
         proc = subprocess.run(args, stdout=subprocess.PIPE, stderr=sys.stderr)
     except FileNotFoundError:
-        raise Exception(f"{args[0]}: Command not found")
+        raise RuntimeError(f"{args[0]}: Command not found")
 
     if proc.returncode == -signal.SIGSEGV:
-        raise Exception(f"{args[0]}: Segmentation fault")
+        raise RuntimeError(f"{args[0]}: Segmentation fault")
 
     if proc.returncode != 0:
-        raise Exception(
-            f"{args[0]}: Process exited with code {proc.returncode}")
+        raise RuntimeError(f"{args[0]}: Process exited with code {proc.returncode}")
 
-    return proc.stdout.decode('utf-8')
+    return proc.stdout.decode("utf-8")
 
 
 def readdir(path: str) -> list[str]:
-    logger.log(f"Reading directory {path}")
+    _logger.info(f"Reading directory {path}")
 
     try:
         return os.listdir(path)
@@ -171,11 +189,40 @@ def readdir(path: str) -> list[str]:
 
 
 def cp(src: str, dst: str):
-    logger.log(f"Copying {src} to {dst}")
+    _logger.info(f"Copying {src} to {dst}")
 
     shutil.copy(src, dst)
 
 
+def mv(src: str, dst: str):
+    _logger.info(f"Moving {src} to {dst}")
+
+    shutil.move(src, dst)
+
+
+def cpTree(src: str, dst: str):
+    _logger.info(f"Copying {src} to {dst}")
+
+    shutil.copytree(src, dst, dirs_exist_ok=True)
+
+
+def cloneDir(url: str, path: str, dest: str) -> str:
+    with tempfile.TemporaryDirectory() as tmp:
+        mkdir(tmp)
+        exec(
+            *["git", "clone", "-n", "--depth=1", "--filter=tree:0", url, tmp, "-q"],
+            quiet=True,
+        )
+        exec(
+            *["git", "-C", tmp, "sparse-checkout", "set", "--no-cone", path, "-q"],
+            quiet=True,
+        )
+        exec(*["git", "-C", tmp, "checkout", "-q", "--no-progress"], quiet=True)
+        mv(os.path.join(tmp, path), dest)
+
+    return dest
+
+
 LATEST_CACHE: dict[str, str] = {}
 
 
@@ -194,9 +241,14 @@ def latest(cmd: str) -> str:
     if cmd in LATEST_CACHE:
         return LATEST_CACHE[cmd]
 
-    logger.log(f"Finding latest version of {cmd}")
+    _logger.info(f"Finding latest version of {cmd}")
 
-    regex = re.compile(r"^" + re.escape(cmd) + r"(-.[0-9]+)?(\.exe)?$")
+    regex: re.Pattern[str]
+
+    if platform.system() == "Windows":
+        regex = re.compile(r"^" + re.escape(cmd) + r"(-.[0-9]+)?(\.exe)?$")
+    else:
+        regex = re.compile(r"^" + re.escape(cmd) + r"(-[0-9]+)?$")
 
     versions: list[str] = []
     for path in os.environ["PATH"].split(os.pathsep):
@@ -206,13 +258,20 @@ def latest(cmd: str) -> str:
                     versions.append(f)
 
     if len(versions) == 0:
-        raise Exception(f"{cmd} not found")
+        raise RuntimeError(f"{cmd} not found")
 
     versions.sort()
     chosen = versions[-1]
 
-    logger.log(f"Chosen {chosen} as latest version of {cmd}")
+    _logger.info(f"Chosen {chosen} as latest version of {cmd}")
 
     LATEST_CACHE[cmd] = chosen
 
     return chosen
+
+
+def which(cmd: str) -> Optional[str]:
+    """
+    Find the path of a command
+    """
+    return shutil.which(cmd)

cutekit/utils.py

@@ -1,22 +1,23 @@
-from typing import Any, TypeVar, cast
+import os
+from typing import Any, TypeVar, cast, Optional, Union
 import json
 import hashlib
 
-from osdk import shell, const
-
-T = TypeVar('T')
+T = TypeVar("T")
 
 
-def uniq(l: list[str]) -> list[str]:
-    result: list[str] = []
-    for i in l:
+def uniq(lst: list[T]) -> list[T]:
+    result: list[T] = []
+    for i in lst:
         if i in result:
             result.remove(i)
         result.append(i)
     return result
 
 
-def hash(obj: Any, keys: list[str] = [], cls: type[json.JSONEncoder] | None = None) -> str:
+def hash(
+    obj: Any, keys: list[str] = [], cls: Optional[type[json.JSONEncoder]] = None
+) -> str:
     toHash = {}
     if len(keys) == 0:
         toHash = obj
@@ -29,7 +30,7 @@ def hash(obj: Any, keys: list[str] = [], cls: type[json.JSONEncoder] | None = No
 
 
 def camelCase(s: str) -> str:
-    s = ''.join(x for x in s.title() if x != '_' and x != '-')
+    s = "".join(x for x in s.title() if x != "_" and x != "-")
     s = s[0].lower() + s[1:]
     return s
 
@@ -52,9 +53,15 @@ def key(obj: Any, keys: list[str] = []) -> str:
     return "-".join(k)
 
 
-def asList(i: T | list[T] | None) -> list[T]:
+def asList(i: Optional[Union[T, list[T]]]) -> list[T]:
     if i is None:
         return []
     if isinstance(i, list):
         return cast(list[T], i)
     return [i]
+
+
+def isNewer(path1: str, path2: str) -> bool:
+    return not os.path.exists(path2) or os.path.getmtime(path1) > os.path.getmtime(
+        path2
+    )

cutekit/vt100.py

@@ -0,0 +1,55 @@
+BLACK = "\033[30m"
+RED = "\033[31m"
+GREEN = "\033[32m"
+BROWN = "\033[33m"
+BLUE = "\033[34m"
+PURPLE = "\033[35m"
+CYAN = "\033[36m"
+WHITE = "\033[37m"
+YELLOW = "\033[33m"
+
+
+BRIGHT_BLACK = "\033[90m"
+BRIGHT_RED = "\033[91m"
+BRIGHT_GREEN = "\033[92m"
+BRIGHT_BROWN = "\033[93m"
+BRIGHT_BLUE = "\033[94m"
+BRIGHT_PURPLE = "\033[95m"
+BRIGHT_CYAN = "\033[96m"
+BRIGHT_WHITE = "\033[97m"
+
+BOLD = "\033[1m"
+FAINT = "\033[2m"
+ITALIC = "\033[3m"
+UNDERLINE = "\033[4m"
+BLINK = "\033[5m"
+NEGATIVE = "\033[7m"
+CROSSED = "\033[9m"
+RESET = "\033[0m"
+
+
+def wordwrap(text: str, width: int = 60, newline: str = "\n") -> str:
+    result = ""
+    curr = 0
+
+    for c in text:
+        if c == " " and curr > width:
+            result += newline
+            curr = 0
+        else:
+            result += c
+            curr += 1
+
+    return result
+
+
+def indent(text: str, indent: int = 4) -> str:
+    return " " * indent + text.replace("\n", "\n" + " " * indent)
+
+
+def title(text: str):
+    print(f"{BOLD}{text}{RESET}:")
+
+
+def p(text: str):
+    return indent(wordwrap(text))

doc/cutekit.md

@@ -0,0 +1,114 @@
+# Cutekit 
+
+Cutekit is a build system that aims to be simple, fast and easy to use.
+A project is described using json files.
+
+## Project file 
+
+The project file is used to describe the project and its dependencies.
+
+See: [doc/spec/project.md](doc/spec/project.md) for the full specification.
+
+Example: 
+
+> project.json
+```json
+{
+    "$schema": "https://schemas.cute.engineering/stable/cutekit.manifest.project.v1",
+    "id": "skift-org/skift",
+    "type": "project",
+    "description": "The Skift Operating System",
+    "extern": {
+        "cute-engineering/libheap": {
+            "git": "https://github.com/cute-engineering/libheap.git",
+            "tag": "v1.1.0"
+        }
+    }
+}
+```
+
+Here we describe a project with the id `skift-org/skift` and a dependency to `cute-engineering/libheap` at version `v1.1.0`.
+
+## An executable package manifest
+
+
+When you want to create an executable package, you need to create a `manifest.json` file in any directory under `src/`.
+This is the file that describe an executable with its dependencies.
+
+> src/nyan-cat-app/manifest.json   
+```json
+{
+    "$schema": "https://schemas.cute.engineering/stable/cutekit.manifest.component.v1",
+    "id": "nyan-cat-app",
+    "type": "exe",
+    "description": "rainbows everywhere",
+    "requires": [
+        "easy-lib"
+    ]
+}
+```
+
+Here we describe an executable with the id `nyan-cat-app` and a dependency to `easy-lib` (which is a library built by the project).
+
+You can run the executable by running `$ ck run nyan-cat-app`.
+
+## A library package manifest
+
+When you want to create a library package, you need to create a `manifest.json` file in any directory under `src/`, like an executable package.
+
+> src/easy-lib/manifest.json   
+```json
+{
+    "$schema": "https://schemas.cute.engineering/stable/cutekit.manifest.component.v1",
+    "id": "easy-lib",
+    "type": "lib",
+    "description": "easy to use library",
+    "requires": [
+        "cute-engineering/libheap"
+    ]
+}
+```
+
+Here we describe a library with the id `easy-lib` and a dependency to `cute-engineering/libheap` (which is an external dependency described above in the `project.json`).
+
+## Using installed libraries
+
+You can create a specific installed library through the use of `pkg-config` files.
+For example here is how you add `SDL2` to your project:
+
+
+> src/extern/sdl2/manifest.json
+```json
+{
+    "$schema": "https://schemas.cute.engineering/stable/cutekit.manifest.component.v1",
+    "id": "sdl2",
+    "type": "lib",
+    "description": "A cross-platform development library designed to provide low level access to hardware",
+    "tools": {
+        "cc": {
+            "args": [
+                "@exec",
+                "pkg-config",
+                "--cflags",
+                "sdl2"
+            ]
+        },
+        "cxx": {
+            "args": [
+                "@exec",
+                "pkg-config",
+                "--cflags",
+                "sdl2"
+            ]
+        },
+        "ld": {
+            "args": [
+                "@exec",
+                "pkg-config",
+                "--libs",
+                "sdl2"
+            ]
+        }
+    }
+}
+```

doc/extends.md

@@ -0,0 +1,18 @@
+# Extending cutekit
+
+By writing custom Python plugins, you can extend Cutekit to do whatever you want.
+
+First the file need to be located in `meta/plugins` and have the `.py` extension.
+Then you can import cutekit and change/add whatever you want.
+
+For example you can add a new command to the CLI:
+
+```python
+from cutekit import cli
+
+@cli.command("h", "hello", "Print hello world")
+def bootCmd(args: cli.Args) -> None:
+    print("Hello world!")
+```
+
+This feature is used - for example - by [SkiftOS](https://github.com/skift-org/skift/blob/main/meta/plugins/start-cmd.py) to add the `start` command, that build packages and run a virtual machine.

doc/mit.svg

@@ -0,0 +1,40 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg width="93.217mm" height="69.497mm" version="1.1" viewBox="0 0 93.217 69.497" xmlns="http://www.w3.org/2000/svg"
+    xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/"
+    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
+    <metadata>
+        <rdf:RDF>
+            <cc:Work rdf:about="">
+                <dc:format>image/svg+xml</dc:format>
+                <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+                <dc:title />
+            </cc:Work>
+        </rdf:RDF>
+    </metadata>
+    <g transform="translate(0 19.497)">
+        <rect y="-19.497" width="93.217" height="69.497" fill="#a00" />
+        <rect x="5.6831" y="-14.441" width="8.8" height="40" ry="0" fill="#fff" />
+        <rect x="5.6332" y="21.559" width="4.45" height="4.05" fill="#a00" />
+        <rect x="10.083" y="17.559" width="4.45" height="4" fill="#a00" />
+        <rect x="5.6332" y="13.559" width="4.45" height="4" fill="#a00" />
+        <rect x="10.083" y="9.5594" width="4.45" height="4" fill="#a00" />
+        <rect x="5.6834" y="-14.441" width="35.2" height="8.8" fill="#fff" />
+        <rect x="32.083" y="-14.441" width="8.8" height="40" ry="0" fill="#fff" />
+        <rect x="36.483" y="21.559" width="4.45" height="4.05" fill="#a00" />
+        <rect x="32.033" y="17.559" width="4.45" height="4" fill="#a00" />
+        <rect x="36.483" y="13.559" width="4.45" height="4" fill="#a00" />
+        <rect x="32.033" y="9.5594" width="4.45" height="4" fill="#a00" />
+        <rect x="20.083" y="-14.441" width="6.6202" height="40" ry="0" fill="#fff" />
+        <rect x="46.083" y="-14.441" width="8.8" height="40" ry="0" fill="#fff" />
+        <rect x="70.083" y="-14.441" width="8.8" height="40" ry="0" fill="#fff" />
+        <rect x="60.283" y="-14.441" width="28.4" height="8.8" fill="#fff" />
+        <rect x="70.033" y="21.559" width="4.45" height="4.05" fill="#a00" />
+        <rect x="74.483" y="17.559" width="4.45" height="4" fill="#a00" />
+        <rect x="70.033" y="13.559" width="4.45" height="4" fill="#a00" />
+        <rect x="74.483" y="9.5594" width="4.45" height="4" fill="#a00" />
+        <text x="12.171427" y="44.462078" fill="#ffffff" font-family="'Liberation Sans'" font-size="11.289px"
+            letter-spacing="0px" stroke-width="1px" word-spacing="0px" style="line-height:125%" xml:space="preserve">
+            <tspan x="12.171427" y="44.462078" font-family="'Liberation Sans'" font-size="19.756px">License</tspan>
+        </text>
+    </g>
+</svg>

doc/spec/jexpr.md

@@ -0,0 +1,92 @@
+
+## Macros
+
+To avoid headache, Cutekit extends JSON through simple macros, this is what we call **Jexpr**.
+
+
+### `@latest`
+
+Find the latest version of a command in the path.
+
+```json
+"cc": {
+    "cmd": ["@latest", "clang"], // clang-14
+/* ... */
+```
+
+
+### `@uname`
+
+Query the system for information about the current operating system.
+
+
+```json
+"cc": {
+    "cmd": ["@uname", "machine"], // "x86_64"
+/* ... */
+```
+
+The `@uname` commands has 1 argument that may be: 
+- `node`: to get the current machine hostname.
+- `machine`: to get the current machine running architecture
+  - `aarch64` is renamed to `arm64`
+  - `AMD64` is renamed to `x86_64`
+- `system`: to get the current machine running operating system: 
+  - `Linux`
+  - `Windows`
+- `release`: to get the current machine operating system's version
+- `version`: to get more information about the host operating system. 
+
+### `@include`
+
+Include a manifest file.
+
+### `@read`
+
+Read a Json file and output its value.
+
+### `@join`
+
+Join two objects
+
+Example:
+
+```json
+["@join",  {"a": 1}, {"b": 2}] // {"a": 1, "b": 2}
+```
+
+### `@concat`
+
+Concatenate strings
+
+Example:
+
+```json
+["@concat", "a", "b", "c"] // "abc"
+```
+
+### `@exec`
+
+Execute a command and return the output
+
+Example:
+
+```json
+["@exec", "uname", "-m"] // "x86_64"
+```
+
+### `@eval`
+
+Execute python code and return the output
+
+Example:
+
+```json
+"32limit": ["@eval", "2**32"]
+```
+
+### `@abspath`
+
+Returns the absolute path of a path.
+
+

doc/spec/manifest.md

@@ -0,0 +1,102 @@
+
+
+## Manifest file format
+
+### `id`
+
+The `id` of the package. This is used to identify the package in the manifest file.
+
+Example:
+
+```json
+{
+    "id": "hello"
+}
+```
+
+### `type`
+
+The type of the package. This is used to identify the package in the manifest file.
+
+Example:
+
+```json
+{
+    "type": "exe"
+}
+```
+
+
+**Values:**
+- `"exe"`
+- `"lib"`
+
+### `description`
+
+The description of the package for the user.
+
+Example:
+
+```json
+{
+    "description": "Hello world"
+}
+```
+
+### `enabledIf`
+
+A list of requirements for the package check against the build props. If the requirement is not met, the package will be disabled.
+
+```json
+{
+    "enabledIf": {
+        "freestanding": [
+            false
+        ]
+    }
+}
+```
+
+**Values:**
+
+`enableIf` is a map of variable and values: 
+```
+"variable-name": [array of expected value] 
+```
+If `variable-name` is equal to one of the value in the table, then the package will be enabled.
+
+
+### `requires`
+
+Dependencies of the package. The name listed here must be the same as the `id` of the package or member of a provide list.
+
+Example:
+
+```json
+{
+    "requires": [
+        "libc",
+        "libm"
+    ]
+}
+```
+
+### `provides`
+
+An alias for the package.
+
+Example:
+
+```json
+{
+    "provides": [
+        "hello"
+    ]
+}
+```
+
+This alias may be used by other package when using `requires`.
+This is used when you have multiple package implementing the same features, but only one is enabled through `enableIf`.
+
+**Value**: 
+- An array of `id`.

doc/spec/project.md

@@ -0,0 +1,33 @@
+## project.json
+
+The project file is the main file of the project.
+It describes the project and its dependencies.
+
+### `id`
+
+The `id` of the project. This is used to identify the project.
+
+### `type`
+
+Should be `project`.
+
+### `description`
+
+The description of the project for the user.
+
+### `extern`
+
+A list of external dependencies for the project, for example: 
+
+```json
+
+"externs": {
+    "cute-engineering/libheap": {
+        "git": "https://github.com/cute-engineering/libheap.git",
+        "tag": "v1.1.0"
+    }
+}
+```
+
+You describe the project `id`, the `git` repository and the `tag` to use.
+

doc/spec/target.md

@@ -0,0 +1,119 @@
+
+## Target file format
+
+A target file is used for describing a new cross-compiler to use.
+Generally, a target file is named like this: `{target}-{arch}.json`. For example, you could have: 
+- `host-x86-64`
+- `windows-arm`
+- ...
+
+### `id`
+
+The `id` of the target. This is used to identify the target in the target file.
+
+### `type`
+
+Should be `target`.
+
+### `props`
+
+A list of properties for the target.
+
+Example:
+
+```json
+{
+    "props": {
+        "arch": "x86_64",
+        "vendor": "pc",
+        "os": "linux",
+        "env": "gnu",
+        "abi": "elf",
+        "cpu": "x86_64",
+        "features": "fxsr,sse,sse2"
+    }
+}
+```
+
+Theses may be accessed during compilation as C-Macros: `__ck_{prop}__`.
+
+### `tools`
+
+A list of tools for the target.
+
+Each tool is described like this: 
+
+```json
+"name": {
+    "cmd": "",
+    "args": [
+        ...
+    ]
+}
+```
+
+Where: 
+- `cmd` describe the command to run
+- `arg` describes each argument to use (note: each entry of the table correspond to ONE argument).
+
+You have different tools names that you can use: 
+- "cc"
+- "cxx"
+- "ld"
+- "ar"
+- "as"
+
+**Example**:
+
+```json
+{
+    "tools": {
+        "cc": {
+            "cmd": ["@latest", "clang"],
+            "args": [
+                "-target",
+                "x86_64-unknown-windows",
+                "-ffreestanding",
+                "-fno-stack-protector",
+                "-fshort-wchar",
+                "-mno-red-zone"
+            ]
+        },
+        "cxx": {
+            "cmd": ["@latest", "clang++"],
+            "args": [
+                "-target",
+                "x86_64-unknown-windows",
+                "-ffreestanding",
+                "-fno-stack-protector",
+                "-fshort-wchar",
+                "-mno-red-zone"
+            ]
+        },
+        "ld": {
+            "cmd": ["@latest", "clang++"],
+            "args": [
+                "-target",
+                "x86_64-unknown-windows",
+                "-nostdlib",
+                "-Wl,-entry:efi_main",
+                "-Wl,-subsystem:efi_application",
+                "-fuse-ld=lld-link"
+            ]
+        },
+        "ar": {
+            "cmd": ["@latest", "llvm-ar"],
+            "args": [
+                "rcs"
+            ]
+        },
+        "as": {
+            "cmd": "nasm",
+            "args": [
+                "-f",
+                "win64"
+            ]
+        }
+    }
+}
+```

doc/templates.md

@@ -0,0 +1,40 @@
+
+# Templates 
+
+Templates are based on this [repository (cute-engineering/cutekit-templates)](https://github.com/cute-engineering/cutekit-templates).
+Each directory correspond to a template.
+
+You can create a new Cutekit project with the `ck I {template-name}` command.
+
+If you want to use another repository as a template, you can use the `ck I --repo="github-link" name` command. For example: 
+
+```bash
+ck I --repo="cute-engineering/cutekit-templates.git" host
+```
+
+## Writing a template
+
+When writing a template, you do it through a github repository (only github for now).
+Then add a `registry.json` file at the root of the repository contaning a table of entry directories.
+
+For example, if you have a UI library called `cute-ui`, you can add a `registry.json` file like this:
+
+```json
+[
+    {
+        "id": "cute-ui-simple",
+        "description": "A simple template"
+    },
+    {
+        "id": "cute-ui-advanced",
+        "description": "A more advanced template"
+    }
+]
+```
+
+And each "id" will correspond to a directory in the repository:
+
+- `cute-ui-simple` will be in `cute-ui-simple/`
+- `cute-ui-advanced` will be in `cute-ui-advanced/`
+
+

osdk/__init__.py

@@ -1,30 +0,0 @@
-import sys
-
-from os.path import isdir
-from osdk import const, shell
-from osdk.args import parse
-from osdk.cmds import exec, usage
-from osdk.plugins import loadAll
-import osdk.vt100 as vt100
-
-
-def main() -> int:
-    a = parse(sys.argv[1:])
-
-    if not a.consumeOpt("verbose", False):
-        if not isdir(const.OSDK_DIR):
-            shell.mkdir(const.OSDK_DIR)
-        sys.stderr = open(f"{const.OSDK_DIR}/osdk.log", "w")
-
-    try:
-        loadAll()
-        exec(a)
-        return 0
-    except Exception as e:
-        print(f"{vt100.RED}{e}{vt100.RESET}")
-        print()
-
-        usage()
-        print()
-
-        raise e

osdk/args.py

@@ -1,56 +0,0 @@
-Value = str | bool | int
-
-
-class Args:
-    opts: dict[str, Value]
-    args: list[str]
-
-    def __init__(self):
-        self.opts = {}
-        self.args = []
-
-    def consumePrefix(self, prefix: str) -> dict[str, Value]:
-        result: dict[str, Value] = {}
-        for key, value in self.opts.items():
-            if key.startswith(prefix):
-                result[key[len(prefix):]] = value
-                del self.opts[key]
-        return result
-
-    def consumeOpt(self, key: str, default: Value) -> Value:
-        if key in self.opts:
-            result = self.opts[key]
-            del self.opts[key]
-            return result
-        return default
-
-    def tryConsumeOpt(self, key: str) -> Value | None:
-        if key in self.opts:
-            result = self.opts[key]
-            del self.opts[key]
-            return result
-        return None
-
-    def consumeArg(self) -> str | None:
-        if len(self.args) == 0:
-            return None
-
-        first = self.args[0]
-        del self.args[0]
-        return first
-
-
-def parse(args: list[str]) -> Args:
-    result = Args()
-
-    for arg in args:
-        if arg.startswith("--"):
-            if "=" in arg:
-                key, value = arg[2:].split("=", 1)
-                result.opts[key] = value
-            else:
-                result.opts[arg[2:]] = True
-        else:
-            result.args.append(arg)
-
-    return result

osdk/builder.py

@@ -1,139 +0,0 @@
-import os
-from typing import TextIO
-
-from osdk.model import ComponentManifest, TargetManifest, Props
-from osdk.ninja import Writer
-from osdk.logger import Logger
-from osdk.context import Context, contextFor
-from osdk import shell, rules
-
-logger = Logger("builder")
-
-
-def gen(out: TextIO, context: Context):
-    writer = Writer(out)
-
-    target = context.target
-
-    writer.comment("File generated by the build system, do not edit")
-    writer.newline()
-    writer.variable("builddir", context.builddir())
-
-    writer.separator("Tools")
-
-    writer.variable("cincs", " ".join(
-        map(lambda i: f"-I{i}", context.cincls())))
-
-    writer.variable("cdefs", " ".join(context.cdefs()))
-
-    writer.newline()
-
-    for i in target.tools:
-        tool = target.tools[i]
-        rule = rules.rules[i]
-        writer.variable(i, tool.cmd)
-        writer.variable(
-            i + "flags", " ".join(rule.args + tool.args))
-        writer.rule(
-            i, f"{tool.cmd} {rule.rule.replace('$flags',f'${i}flags')}", depfile=rule.deps)
-        writer.newline()
-
-    writer.separator("Components")
-
-    all: list[str] = []
-
-    for instance in context.enabledInstances():
-        objects = instance.objsfiles(context)
-        writer.comment(f"Component: {instance.manifest.id}")
-        writer.comment(f"Resolved: {', '.join(instance.resolved)}")
-
-        for obj in objects:
-            r = rules.byFileIn(obj[0])
-            if r is None:
-                raise Exception(f"Unknown rule for file {obj[0]}")
-            t = target.tools[r.id]
-            writer.build(obj[1], r.id,  obj[0], order_only=t.files)
-
-        writer.newline()
-
-        if instance.isLib():
-            writer.build(instance.libfile(context), "ar",
-                         list(map(lambda o: o[1], objects)))
-        else:
-            libraries: list[str] = []
-
-            for req in instance.resolved:
-                reqInstance = context.componentByName(req)
-
-                if reqInstance is None:
-                    raise Exception(f"Component {req} not found")
-
-                if not reqInstance.isLib():
-                    raise Exception(f"Component {req} is not a library")
-
-                libraries.append(reqInstance.outfile(context))
-
-            writer.build(instance.binfile(context), "ld",
-                         list(map(lambda o: o[1], objects)) + libraries)
-
-            all.append(instance.binfile(context))
-
-        writer.newline()
-
-    writer.separator("Phony targets")
-
-    writer.build("all", "phony", all)
-    writer.default("all")
-
-
-def build(componentSpec: str, targetSpec: str, props: Props = {}) -> str:
-    context = contextFor(targetSpec, props)
-
-    shell.mkdir(context.builddir())
-    ninjaPath = os.path.join(context.builddir(), "build.ninja")
-
-    with open(ninjaPath, "w") as f:
-        gen(f, context)
-
-    instance = context.componentByName(componentSpec)
-
-    if instance is None:
-        raise Exception(f"Component {componentSpec} not found")
-
-    if not instance.enabled:
-        raise Exception(
-            f"Component {componentSpec} is disabled: {instance.disableReason}")
-
-    shell.exec(f"ninja", "-v", "-f", ninjaPath, instance.outfile(context))
-
-    return instance.outfile(context)
-
-
-class Paths:
-    bin: str
-    lib: str
-    obj: str
-
-    def __init__(self, bin: str, lib: str, obj: str):
-        self.bin = bin
-        self.lib = lib
-        self.obj = obj
-
-
-def buildAll(targetSpec: str) -> Paths:
-    context = contextFor(targetSpec)
-    target = context.target
-
-    shell.mkdir(context.builddir())
-    ninjaPath = os.path.join(context.builddir(), "build.ninja")
-
-    with open(ninjaPath, "w") as f:
-        gen(f, context)
-
-    shell.exec(f"ninja", "-v", "-f", ninjaPath)
-
-    return Paths(
-        os.path.join(context.builddir(), "bin"),
-        os.path.join(context.builddir(), "lib"),
-        os.path.join(context.builddir(), "obj")
-    )

osdk/cmds.py

@@ -1,347 +0,0 @@
-from typing import Callable, cast
-import os
-import json
-
-from osdk.args import Args
-from osdk.context import contextFor
-from osdk import context, shell, const, vt100, builder, graph
-
-Callback = Callable[[Args], None]
-
-
-class Cmd:
-    shortName: str | None
-    longName: str
-    helpText: str
-    callback: Callable[[Args], None]
-    isPlugin: bool = False
-
-    def __init__(self, shortName: str | None, longName: str, helpText: str, callback: Callable[[Args], None]):
-        self.shortName = shortName
-        self.longName = longName
-        self.helpText = helpText
-        self.callback = callback
-
-
-cmds: list[Cmd] = []
-
-
-def append(cmd: Cmd):
-    cmd.isPlugin = True
-    cmds.append(cmd)
-    cmds.sort(key=lambda c: c.shortName or c.longName)
-
-
-def runCmd(args: Args):
-    targetSpec = cast(str, args.consumeOpt(
-        "target", "host-" + shell.uname().machine))
-
-    componentSpec = args.consumeArg()
-
-    if componentSpec is None:
-        raise Exception("Component not specified")
-
-    exe = builder.build(componentSpec, targetSpec)
-
-    shell.exec(exe)
-
-
-cmds += [Cmd("r", "run", "Run the target", runCmd)]
-
-
-def debugCmd(args: Args):
-    targetSpec = cast(str, args.consumeOpt(
-        "target", "host-" + shell.uname().machine))
-
-    componentSpec = args.consumeArg()
-
-    if componentSpec is None:
-        raise Exception("Component not specified")
-
-    exe = builder.build(componentSpec, targetSpec)
-
-    shell.exec("lldb", "-o", "run", exe)
-
-
-cmds += [Cmd("d", "debug", "Debug the target", debugCmd)]
-
-
-def buildCmd(args: Args):
-    targetSpec = cast(str, args.consumeOpt(
-        "target", "host-" + shell.uname().machine))
-
-    componentSpec = args.consumeArg()
-
-    if componentSpec is None:
-        builder.buildAll(targetSpec)
-    else:
-        builder.build(componentSpec, targetSpec)
-
-
-cmds += [Cmd("b", "build", "Build the target", buildCmd)]
-
-
-def listCmd(args: Args):
-    components = context.loadAllComponents()
-    targets = context.loadAllTargets()
-
-    vt100.title("Components")
-    if len(components) == 0:
-        print(f"   (No components available)")
-    else:
-        print(vt100.indent(vt100.wordwrap(
-            ", ".join(map(lambda m: m.id, components)))))
-    print()
-
-    vt100.title("Targets")
-
-    if len(targets) == 0:
-        print(f"   (No targets available)")
-    else:
-        print(vt100.indent(vt100.wordwrap(", ".join(map(lambda m: m.id, targets)))))
-
-    print()
-
-
-cmds += [Cmd("l", "list", "List the targets", listCmd)]
-
-
-def cleanCmd(args: Args):
-    shell.rmrf(const.BUILD_DIR)
-
-
-cmds += [Cmd("c", "clean", "Clean the build directory", cleanCmd)]
-
-
-def nukeCmd(args: Args):
-    shell.rmrf(const.OSDK_DIR)
-
-
-cmds += [Cmd("n", "nuke", "Clean the build directory and cache", nukeCmd)]
-
-
-def helpCmd(args: Args):
-    usage()
-
-    print()
-
-    vt100.title("Description")
-    print("    Operating System Development Kit.")
-
-    print()
-    vt100.title("Commands")
-    for cmd in cmds:
-        pluginText = ""
-        if cmd.isPlugin:
-            pluginText = f"{vt100.CYAN}(plugin){vt100.RESET}"
-
-        print(
-            f" {vt100.GREEN}{cmd.shortName or ' '}{vt100.RESET}  {cmd.longName} - {cmd.helpText} {pluginText}")
-
-    print()
-
-
-cmds += [Cmd("h", "help", "Show this help message", helpCmd)]
-
-
-def versionCmd(args: Args):
-    print(f"OSDK v{const.get_version()}\n")
-
-
-cmds += [Cmd("v", "version", "Show current version", versionCmd)]
-
-
-def graphCmd(args: Args):
-    targetSpec = cast(str, args.consumeOpt(
-        "target", "host-" + shell.uname().machine))
-
-    scope: str | None = cast(str | None, args.tryConsumeOpt("scope"))
-    onlyLibs: bool = args.consumeOpt("only-libs", False) == True
-    showDisabled: bool = args.consumeOpt("show-disabled", False) == True
-
-    context = contextFor(targetSpec)
-
-    graph.view(context, scope=scope, showExe=not onlyLibs,
-               showDisabled=showDisabled)
-
-
-cmds += [Cmd("g", "graph", "Show dependency graph", graphCmd)]
-
-
-def installCmd(args: Args):
-    project = context.loadProject(".")
-
-    for extSpec in project.extern:
-        ext = project.extern[extSpec]
-
-        extPath = os.path.join(const.EXTERN_DIR, extSpec)
-
-        if os.path.exists(extPath):
-            print(f"Skipping {extSpec}, already installed")
-            continue
-
-        print(f"Installing {extSpec}-{ext.tag} from {ext.git}...")
-        shell.popen("git", "clone", "--depth", "1", "--branch",
-                    ext.tag, ext.git, extPath)
-
-
-cmds += [Cmd("i", "install", "Install all the external packages", installCmd)]
-
-
-def initCmd(args: Args):
-    """
-    |
-    | - project.json
-    | - src/
-    |   | - project_name/
-    |       | - main.c
-    |       | - manifest.json
-    | - meta/
-    |   | - targets/
-    |   |   | - host-*.json
-    |   | - plugins/
-    |   |   | - run.py
-    | - .gitignore
-    | - README.md
-    |
-    """
-
-    print("This utility will walk you through creating a new project.")
-    print("This only covers the most common items, and tries to give sensible defaults.")
-    print()
-    print("First, let's create a project.json file.")
-
-    project_name = input("Project name: ")
-    description = input("Description: ")
-
-    to_create = ["src", "meta", os.path.join("meta", "targets"), os.path.join("meta", "plugins")]
-
-    os.mkdir(project_name.lower())
-    for directory in to_create:
-        os.mkdir(os.path.join(project_name.lower(), directory))
-
-    with open(os.path.join(project_name.lower(), "project.json"), "w") as f:
-        f.write(json.dumps({
-            "$schema": "https://schemas.cute.engineering/latest/osdk.manifest.component",
-            "name": project_name,
-            "type": "project",
-            "description": description,
-            "extern": {},
-        }, indent=4))
-
-    with open(os.path.join(project_name.lower(), ".gitignore"), "w") as f:
-        f.write(".osdk\n.ninja_log\n__pycache__\n")
-
-    with open(os.path.join(project_name.lower(), "README.md"), "w") as f:
-        f.write(f"# {project_name}\n")
-        f.write("I was created using the OSDK!\n")
-        f.write(
-            "You can find more information about the OSDK in its [Repo](https://github.com/cute-engineering/osdk)."
-        )
-
-    with open(os.path.join(project_name.lower(), "src", "main.c"), "w") as f:
-        f.write("#include <stdio.h>\n\n")
-        f.write("int main(void)\n{\n")
-        f.write("    printf(\"Hello, World!\\n\");\n")
-        f.write("    return 0;\n")
-        f.write("}")
-
-    with open(os.path.join(project_name.lower(), "src", "manifest.json"), "w") as f:
-        f.write(json.dumps({
-            "$schema": "https://schemas.cute.engineering/latest/osdk.manifest.component",
-            "id": project_name.lower(),
-            "type": "exe",
-            "description": description,
-        }, indent=4))
-
-    with open(os.path.join(project_name.lower(), "meta", "plugins", "run.py"), "w") as f:
-        f.write("from osdk import builder, shell\n")
-        f.write("from osdk.args import Args\n")
-        f.write("from osdk.cmds import Cmd, append\n\n")
-        f.write("def runCmd(args: Args) -> None:\n")
-        f.write(
-            f"    {project_name.lower()} = builder.build(\"{project_name.lower()}\", \"host-{shell.uname().machine}\")\n"
-        )
-        f.write(f"    shell.exec(*[{project_name.lower()}])")
-        f.write("\n\nappend(Cmd(\"s\", \"start\", \"Run the project\", runCmd))")
-
-    with open(os.path.join(project_name.lower(), "meta", "targets", f"host-{shell.uname().machine}.json"), "w") as f:
-        f.write(json.dumps({
-            "$schema": "https://schemas.cute.engineering/latest/osdk.manifest.component",
-            "id": f"host-{shell.uname().machine}",
-            "type": "target",
-            "props": {
-                "arch": shell.uname().machine,
-                "toolchain": "clang",
-                "sys": [
-                    "@uname",
-                    "sysname"
-                ],
-                "abi": "unknown",
-                "freestanding": False,
-                "host": True,
-            },
-            "tools": {
-                "cc": {
-                    "cmd": [
-                        "@latest",
-                        "clang"
-                    ],
-                    "args": []
-                },
-                "cxx": {
-                    "cmd": [
-                        "@latest",
-                        "clang++"
-                    ],
-                    "args": []
-                },
-                "ld": {
-                    "cmd": [
-                        "@latest",
-                        "clang++"
-                    ],
-                    "args": [
-                    ]
-                },
-                "ar": {
-                    "cmd": [
-                        "@latest",
-                        "llvm-ar"
-                    ],
-                    "args": [
-                        "rcs"
-                    ]
-                },
-                "as": {
-                    "cmd": "clang",
-                    "args": [
-                        "-c"
-                    ]
-                }
-            }
-        }, indent=4))
-
-        shell.exec(*["git", "init", project_name.lower()])
-        print("Done! Don't forget to add a LICENSE ;)")
-
-
-cmds += [Cmd("I", "init", "Start a new project", initCmd)]
-
-
-def usage():
-    print(f"Usage: {const.ARGV0} <command> [args...]")
-
-
-def exec(args: Args):
-    cmd = args.consumeArg()
-
-    if cmd is None:
-        raise Exception("No command specified")
-
-    for c in cmds:
-        if c.shortName == cmd or c.longName == cmd:
-            c.callback(args)
-            return
-
-    raise Exception(f"Unknown command {cmd}")

osdk/const.py

@@ -1,14 +0,0 @@
-import os
-import sys
-import subprocess
-
-VERSION = "0.4.1"
-MODULE_DIR = os.path.dirname(os.path.realpath(__file__))
-ARGV0 = os.path.basename(sys.argv[0])
-OSDK_DIR = ".osdk"
-BUILD_DIR = os.path.join(OSDK_DIR, "build")
-CACHE_DIR = os.path.join(OSDK_DIR, "cache")
-EXTERN_DIR = os.path.join(OSDK_DIR, "extern")
-SRC_DIR = "src"
-META_DIR = f"meta"
-TARGETS_DIR = os.path.join(META_DIR, "targets")

osdk/context.py

@@ -1,264 +0,0 @@
-from typing import cast, Protocol, Iterable
-from itertools import chain
-from pathlib import Path
-import os
-
-
-from osdk.model import ProjectManifest, TargetManifest, ComponentManifest, Props, Type, Tool, Tools
-from osdk.logger import Logger
-from osdk import const, shell, jexpr, utils, rules, mixins
-
-logger = Logger("context")
-
-
-class IContext(Protocol):
-    def builddir(self) -> str:
-        ...
-
-
-class ComponentInstance:
-    enabled: bool = True
-    disableReason = ""
-    manifest: ComponentManifest
-    sources: list[str] = []
-    resolved: list[str] = []
-
-    def __init__(
-            self,
-            enabled: bool,
-            disableReason: str,
-            manifest: ComponentManifest,
-            sources: list[str],
-            resolved: list[str]):
-        self.enabled = enabled
-        self.disableReason = disableReason
-        self.manifest = manifest
-        self.sources = sources
-        self.resolved = resolved
-
-    def isLib(self):
-        return self.manifest.type == Type.LIB
-
-    def binfile(self, context: IContext) -> str:
-        return os.path.join(context.builddir(), "bin", f"{self.manifest.id}.out")
-
-    def objdir(self, context: IContext) -> str:
-        return os.path.join(context.builddir(), "obj", self.manifest.id)
-
-    def objsfiles(self, context: IContext) -> list[tuple[str, str]]:
-        return list(
-            map(
-                lambda s: (
-                    s, os.path.join(self.objdir(context), s.replace(os.path.join(self.manifest.dirname(), ''), '') + ".o")),
-                self.sources))
-
-    def libfile(self, context: IContext) -> str:
-        return os.path.join(context.builddir(), "lib", f"{self.manifest.id}.a")
-
-    def outfile(self, context: IContext) -> str:
-        if self.isLib():
-            return self.libfile(context)
-        else:
-            return self.binfile(context)
-
-    def cinclude(self) -> str:
-        if "cpp-root-include" in self.manifest.props:
-            return self.manifest.dirname()
-        else:
-            return str(Path(self.manifest.dirname()).parent)
-
-
-class Context(IContext):
-    target: TargetManifest
-    instances: list[ComponentInstance]
-    tools: Tools
-
-    def enabledInstances(self) -> Iterable[ComponentInstance]:
-        return filter(lambda x: x.enabled, self.instances)
-
-    def __init__(self, target: TargetManifest, instances: list[ComponentInstance], tools: Tools):
-        self.target = target
-        self.instances = instances
-        self.tools = tools
-
-    def componentByName(self, name: str) -> ComponentInstance | None:
-        result = list(filter(lambda x: x.manifest.id == name, self.instances))
-        if len(result) == 0:
-            return None
-        return result[0]
-
-    def cincls(self) -> list[str]:
-        includes = list(
-            map(lambda x: x.cinclude(), self.enabledInstances()))
-        return utils.uniq(includes)
-
-    def cdefs(self) -> list[str]:
-        return self.target.cdefs()
-
-    def hashid(self) -> str:
-        return utils.hash((self.target.props, str(self.tools)))[0:8]
-
-    def builddir(self) -> str:
-        return os.path.join(const.BUILD_DIR, f"{self.target.id}-{self.hashid()[:8]}")
-
-
-def loadAllTargets() -> list[TargetManifest]:
-    files = shell.find(const.TARGETS_DIR, ["*.json"])
-    return list(
-        map(lambda path: TargetManifest(jexpr.evalRead(path), path), files))
-
-
-def loadProject(path: str) -> ProjectManifest:
-    path = os.path.join(path, "project.json")
-    return ProjectManifest(jexpr.evalRead(path), path)
-
-
-def loadTarget(id: str) -> TargetManifest:
-    try:
-        return next(filter(lambda t: t.id == id, loadAllTargets()))
-    except StopIteration:
-        raise Exception(f"Target '{id}' not found")
-
-
-def loadAllComponents() -> list[ComponentManifest]:
-    files = shell.find(const.SRC_DIR, ["manifest.json"])
-    files += shell.find(const.EXTERN_DIR, ["manifest.json"])
-
-    return list(
-        map(
-            lambda path: ComponentManifest(jexpr.evalRead(path), path),
-            files))
-
-
-def filterDisabled(components: list[ComponentManifest], target: TargetManifest) -> tuple[list[ComponentManifest], list[ComponentManifest]]:
-    return list(filter(lambda c: c.isEnabled(target)[0], components)), \
-        list(filter(lambda c: not c.isEnabled(target)[0], components))
-
-
-def providerFor(what: str, components: list[ComponentManifest]) -> tuple[str | None, str]:
-    result: list[ComponentManifest] = list(
-        filter(lambda c: c.id == what, components))
-
-    if len(result) == 0:
-        # Try to find a provider
-        result = list(filter(lambda x: (what in x.provides), components))
-
-    if len(result) == 0:
-        logger.error(f"No provider for '{what}'")
-        return (None, f"No provider for '{what}'")
-
-    if len(result) > 1:
-        ids = list(map(lambda x: x.id, result))
-        logger.error(f"Multiple providers for '{what}': {result}")
-        return (None, f"Multiple providers for '{what}': {','.join(ids)}")
-
-    return (result[0].id, "")
-
-
-def resolveDeps(componentSpec: str, components: list[ComponentManifest], target: TargetManifest) -> tuple[bool, str,  list[str]]:
-    mapping = dict(map(lambda c: (c.id, c), components))
-
-    def resolveInner(what: str, stack: list[str] = []) -> tuple[bool, str,  list[str]]:
-        result: list[str] = []
-        what = target.route(what)
-        resolved, unresolvedReason = providerFor(what, components)
-
-        if resolved is None:
-            return False, unresolvedReason,  []
-
-        if resolved in stack:
-            raise Exception(f"Dependency loop: {stack} -> {resolved}")
-
-        stack.append(resolved)
-
-        for req in mapping[resolved].requires:
-            keep, unresolvedReason,  reqs = resolveInner(req, stack)
-
-            if not keep:
-                stack.pop()
-                logger.error(f"Dependency '{req}' not met for '{resolved}'")
-                return False, unresolvedReason,  []
-
-            result.extend(reqs)
-
-        stack.pop()
-        result.insert(0, resolved)
-
-        return True, "", result
-
-    enabled, unresolvedReason, resolved = resolveInner(componentSpec)
-
-    return enabled, unresolvedReason, resolved
-
-
-def instanciate(componentSpec: str, components: list[ComponentManifest], target: TargetManifest) -> ComponentInstance | None:
-    manifest = next(filter(lambda c: c.id == componentSpec, components))
-    wildcards = set(
-        chain(*map(lambda rule: rule.fileIn, rules.rules.values())))
-    sources = shell.find(
-        manifest.subdirs, list(wildcards), recusive=False)
-    enabled, unresolvedReason, resolved = resolveDeps(
-        componentSpec, components, target)
-
-    return ComponentInstance(enabled, unresolvedReason, manifest, sources, resolved[1:])
-
-
-def instanciateDisabled(component: ComponentManifest,  target: TargetManifest) -> ComponentInstance:
-    return ComponentInstance(False, component.isEnabled(target)[1], component, [], [])
-
-
-context: dict = {}
-
-
-def contextFor(targetSpec: str, props: Props = {}) -> Context:
-    if targetSpec in context:
-        return context[targetSpec]
-
-    logger.log(f"Loading context for '{targetSpec}'")
-
-    targetEls = targetSpec.split(":")
-
-    if targetEls[0] == "":
-        targetEls[0] = "host-" + shell.uname().machine
-
-    target = loadTarget(targetEls[0])
-    target.props |= props
-
-    components = loadAllComponents()
-    components, disabled = filterDisabled(components, target)
-
-    tools: Tools = {}
-
-    for toolSpec in target.tools:
-        tool = target.tools[toolSpec]
-
-        tools[toolSpec] = Tool(
-            strict=False,
-            cmd=tool.cmd,
-            args=tool.args,
-            files=tool.files)
-
-        tools[toolSpec].args += rules.rules[toolSpec].args
-
-    for m in targetEls[1:]:
-        mixin = mixins.byId(m)
-        tools = mixin(target, tools)
-
-    for component in components:
-        for toolSpec in component.tools:
-            tool = component.tools[toolSpec]
-            tools[toolSpec].args += tool.args
-
-    instances: list[ComponentInstance] = list(
-        map(lambda c: instanciateDisabled(c, target), disabled))
-
-    instances += cast(list[ComponentInstance], list(filter(lambda e: e != None, map(lambda c: instanciate(
-        c.id, components, target), components))))
-
-    context[targetSpec] = Context(
-        target,
-        instances,
-        tools,
-    )
-
-    return context[targetSpec]

osdk/graph.py

@@ -1,58 +0,0 @@
-import os
-
-from osdk.context import Context
-from osdk import vt100
-
-
-def view(context: Context, scope: str | None = None, showExe: bool = True, showDisabled: bool = False):
-    from graphviz import Digraph
-
-    g = Digraph(context.target.id, filename='graph.gv')
-
-    g.attr('graph',  splines='ortho', rankdir='BT', ranksep='1.5')
-    g.attr('node', shape='ellipse')
-    g.attr(
-        'graph', label=f"<<B>{scope or 'Full Dependency Graph'}</B><BR/>{context.target.id}>", labelloc='t')
-
-    scopeInstance = None
-
-    if scope is not None:
-        scopeInstance = context.componentByName(scope)
-
-    for instance in context.instances:
-        if not instance.isLib() and not showExe:
-            continue
-
-        if scopeInstance is not None and \
-            instance.manifest.id != scope and \
-                instance.manifest.id not in scopeInstance.resolved:
-            continue
-
-        if instance.enabled:
-            fillcolor = "lightgrey" if instance.isLib() else "lightblue"
-            shape = "plaintext" if not scope == instance.manifest.id else 'box'
-
-            g.node(instance.manifest.id, f"<<B>{instance.manifest.id}</B><BR/>{vt100.wordwrap(instance.manifest.decription, 40,newline='<BR/>')}>",
-                   shape=shape, style="filled", fillcolor=fillcolor)
-
-            for req in instance.manifest.requires:
-                g.edge(instance.manifest.id, req)
-
-            for req in instance.manifest.provides:
-                isChosen = context.target.routing.get(
-                    req, None) == instance.manifest.id
-
-                g.edge(req, instance.manifest.id, arrowhead="none", color=(
-                    "blue" if isChosen else "black"))
-        elif showDisabled:
-            g.node(instance.manifest.id, f"<<B>{instance.manifest.id}</B><BR/>{vt100.wordwrap(instance.manifest.decription, 40,newline='<BR/>')}<BR/><BR/><I>{vt100.wordwrap(instance.disableReason, 40,newline='<BR/>')}</I>>",
-                   shape="plaintext", style="filled",   fontcolor="#999999", fillcolor="#eeeeee")
-
-            for req in instance.manifest.requires:
-                g.edge(instance.manifest.id, req, color="#aaaaaa")
-
-            for req in instance.manifest.provides:
-                g.edge(req, instance.manifest.id,
-                       arrowhead="none", color="#aaaaaa")
-
-    g.view(filename=os.path.join(context.builddir(), "graph.gv"))

osdk/jexpr.py

@@ -1,51 +0,0 @@
-from typing import Any, cast, Callable, Final
-import json
-
-import osdk.shell as shell
-
-Json = Any
-Builtin = Callable[..., Json]
-
-BUILTINS: Final[dict[str, Builtin]] = {
-    "uname": lambda arg: getattr(shell.uname(), arg).lower(),
-    "include": lambda arg: evalRead(arg),
-    "evalRead": lambda arg: evalRead(arg),
-    "join": lambda lhs, rhs: cast(Json, {**lhs, **rhs} if isinstance(lhs, dict) else lhs + rhs),
-    "concat": lambda *args: "".join(args),
-    "eval": lambda arg: eval(arg),
-    "read": lambda arg: read(arg),
-    "exec": lambda *args: shell.popen(*args).splitlines(),
-    "latest": lambda arg: shell.latest(arg),
-}
-
-
-def eval(jexpr: Json) -> Json:
-    if isinstance(jexpr, dict):
-        result = {}
-        for k in cast(dict[str, Json], jexpr):
-            result[k] = eval(jexpr[k])
-        return cast(Json, result)
-    elif isinstance(jexpr, list):
-        jexpr = cast(list[Json], jexpr)
-        if len(jexpr) > 0 and isinstance(jexpr[0], str) and jexpr[0].startswith("@"):
-            funcName = jexpr[0][1:]
-            if funcName in BUILTINS:
-                return BUILTINS[funcName](*eval(jexpr[1:]))
-
-            raise Exception(f"Unknown macro {funcName}")
-        else:
-            return list(map(eval, jexpr))
-    else:
-        return jexpr
-
-
-def read(path: str) -> Json:
-    try:
-        with open(path, "r") as f:
-            return json.load(f)
-    except:
-        raise Exception(f"Failed to read {path}")
-
-
-def evalRead(path: str) -> Json:
-    return eval(read(path))

osdk/logger.py

@@ -1,21 +0,0 @@
-import sys
-import osdk.vt100 as vt100
-
-
-class Logger:
-    name: str
-
-    def __init__(self, name: str):
-        self.name = name
-
-    def log(self, message: str):
-        print(
-            f"{vt100.CYAN}[{self.name}]{vt100.RESET} {message}", file=sys.stderr)
-
-    def warn(self, message: str):
-        print(
-            f"{vt100.YELLOW}[{self.name}]{vt100.RESET} {message}", file=sys.stderr)
-
-    def error(self, message: str):
-        print(
-            f"{vt100.RED}[{self.name}]{vt100.RESET} {message}", file=sys.stderr)

osdk/mixins.py

@@ -1,70 +0,0 @@
-from typing import Callable
-from osdk.model import TargetManifest, Tools
-
-Mixin = Callable[[TargetManifest, Tools], Tools]
-
-
-def patchToolArgs(tools: Tools, toolSpec: str, args: list[str]):
-    tools[toolSpec].args += args
-
-
-def prefixToolCmd(tools: Tools, toolSpec: str, prefix: str):
-    tools[toolSpec].cmd = prefix + " " + tools[toolSpec].cmd
-
-
-def mixinCache(target: TargetManifest, tools: Tools) -> Tools:
-    prefixToolCmd(tools, "cc", "ccache")
-    prefixToolCmd(tools, "cxx", "ccache")
-    return tools
-
-
-def makeMixinSan(san: str) -> Mixin:
-    def mixinSan(target: TargetManifest, tools: Tools) -> Tools:
-        patchToolArgs(
-            tools, "cc", [f"-fsanitize={san}"])
-        patchToolArgs(
-            tools, "cxx", [f"-fsanitize={san}"])
-        patchToolArgs(
-            tools, "ld", [f"-fsanitize={san}"])
-
-        return tools
-
-    return mixinSan
-
-
-def makeMixinOptimize(level: str) -> Mixin:
-    def mixinOptimize(target: TargetManifest, tools: Tools) -> Tools:
-        patchToolArgs(tools, "cc", [f"-O{level}"])
-        patchToolArgs(tools, "cxx", [f"-O{level}"])
-
-        return tools
-
-    return mixinOptimize
-
-
-def mixinDebug(target: TargetManifest, tools: Tools) -> Tools:
-    patchToolArgs(tools, "cc", ["-g"])
-    patchToolArgs(tools, "cxx", ["-g"])
-
-    return tools
-
-
-mixins: dict[str, Mixin] = {
-    "cache": mixinCache,
-    "debug": mixinDebug,
-    "asan": makeMixinSan("address"),
-    "msan": makeMixinSan("memory"),
-    "tsan": makeMixinSan("thread"),
-    "ubsan": makeMixinSan("undefined"),
-    "o3": makeMixinOptimize("3"),
-    "o2": makeMixinOptimize("2"),
-    "o1": makeMixinOptimize("1"),
-    "o0": makeMixinOptimize("0"),
-}
-
-
-def append(mixinSpec: str, mixin: Mixin):
-    mixins[mixinSpec] = mixin
-
-def byId(id: str) -> Mixin:
-    return mixins[id]

osdk/model.py

@@ -1,227 +0,0 @@
-import os
-from enum import Enum
-from typing import Any
-from json import JSONEncoder
-
-from osdk.jexpr import Json
-from osdk.logger import Logger
-from osdk import const, utils
-
-
-logger = Logger("model")
-
-Props = dict[str, Any]
-
-
-class Type(Enum):
-    UNKNOWN = "unknown"
-    PROJECT = "project"
-    TARGET = "target"
-    LIB = "lib"
-    EXE = "exe"
-
-
-class Manifest:
-    id: str = ""
-    type: Type = Type.UNKNOWN
-    path: str = ""
-
-    def __init__(self,  json: Json = None, path: str = "", strict=True, **kwargs):
-        if json is not None:
-            if not "id" in json:
-                raise ValueError("Missing id")
-
-            self.id = json["id"]
-
-            if not "type" in json and strict:
-                raise ValueError("Missing type")
-
-            self.type = Type(json["type"])
-
-            self.path = path
-        elif strict:
-            raise ValueError("Missing json")
-
-        for key in kwargs:
-            setattr(self, key, kwargs[key])
-
-    def __str__(self):
-        return f"Manifest(id={self.id}, type={self.type}, path={self.path})"
-
-    def __repr__(self):
-        return f"Manifest({id})"
-
-    def dirname(self) -> str:
-        return os.path.dirname(self.path)
-
-
-class Extern:
-    git: str = ""
-    tag: str = ""
-
-    def __init__(self, json: Json = None, strict=True, **kwargs):
-        if json is not None:
-            if not "git" in json and strict:
-                raise ValueError("Missing git")
-
-            self.git = json["git"]
-
-            if not "tag" in json and strict:
-                raise ValueError("Missing tag")
-
-            self.tag = json["tag"]
-        elif strict:
-            raise ValueError("Missing json")
-
-        for key in kwargs:
-            setattr(self, key, kwargs[key])
-
-    def __str__(self):
-        return f"Extern(git={self.git}, tag={self.tag})"
-
-    def __repr__(self):
-        return f"Extern({self.git})"
-
-
-class ProjectManifest(Manifest):
-    description: str = ""
-    extern: dict[str, Extern] = {}
-
-    def __init__(self, json: Json = None, path: str = "", strict=True, **kwargs):
-        if json is not None:
-            if not "description" in json and strict:
-                raise ValueError("Missing description")
-
-            self.description = json["description"]
-
-            self.extern = {k: Extern(v)
-                           for k, v in json.get("extern", {}).items()}
-        elif strict:
-            raise ValueError("Missing json")
-
-        super().__init__(json, path, strict, **kwargs)
-
-    def __str__(self):
-        return f"ProjectManifest(id={self.id}, type={self.type}, path={self.path}, description={self.description}, extern={self.extern})"
-
-    def __repr__(self):
-        return f"ProjectManifest({self.id})"
-
-
-class Tool:
-    cmd: str = ""
-    args: list[str] = []
-    files: list[str] = []
-
-    def __init__(self, json: Json = None, strict=True, **kwargs):
-        if json is not None:
-            if not "cmd" in json and strict:
-                raise ValueError("Missing cmd")
-
-            self.cmd = json.get("cmd", self.cmd)
-
-            if not "args" in json and strict:
-                raise ValueError("Missing args")
-
-            self.args = json.get("args", [])
-
-            self.files = json.get("files", [])
-        elif strict:
-            raise ValueError("Missing json")
-
-        for key in kwargs:
-            setattr(self, key, kwargs[key])
-
-    def __str__(self):
-        return f"Tool(cmd={self.cmd}, args={self.args}, files={self.files})"
-
-    def __repr__(self):
-        return f"Tool({self.cmd})"
-
-
-Tools = dict[str, Tool]
-
-
-class TargetManifest(Manifest):
-    props: Props
-    tools: Tools
-    routing: dict[str, str]
-
-    def __init__(self, json: Json = None, path: str = "", strict=True, **kwargs):
-        if json is not None:
-            if not "props" in json and strict:
-                raise ValueError("Missing props")
-
-            self.props = json["props"]
-
-            if not "tools" in json and strict:
-                raise ValueError("Missing tools")
-
-            self.tools = {k: Tool(v) for k, v in json["tools"].items()}
-
-            self.routing = json.get("routing", {})
-
-        super().__init__(json, path, strict, **kwargs)
-
-    def __repr__(self):
-        return f"TargetManifest({self.id})"
-
-    def route(self, componentSpec: str):
-        return self.routing[componentSpec] if componentSpec in self.routing else componentSpec
-
-    def cdefs(self) -> list[str]:
-        defines: list[str] = []
-
-        for key in self.props:
-            macroname = key.lower().replace("-", "_")
-            prop = self.props[key]
-            macrovalue = str(prop).lower().replace(" ", "_").replace("-", "_")
-            if isinstance(prop, bool):
-                if prop:
-                    defines += [f"-D__osdk_{macroname}__"]
-            else:
-                defines += [f"-D__osdk_{macroname}_{macrovalue}__"]
-
-        return defines
-
-
-class ComponentManifest(Manifest):
-    decription: str = "(No description)"
-    props: Props = {}
-    tools: Tools = {}
-    enableIf: dict[str, list[Any]] = {}
-    requires: list[str] = []
-    provides: list[str] = []
-    subdirs: list[str] = []
-
-    def __init__(self, json: Json = None, path: str = "", strict=True, **kwargs):
-        if json is not None:
-            self.decription = json.get("description", self.decription)
-            self.props = json.get("props", self.props)
-            self.tools = {k: Tool(v, strict=False)
-                          for k, v in json.get("tools", {}).items()}
-            self.enableIf = json.get("enableIf", self.enableIf)
-            self.requires = json.get("requires", self.requires)
-            self.provides = json.get("provides", self.provides)
-            self.subdirs = list(map(lambda x: os.path.join(os.path.dirname(
-                path), x), json.get("subdirs", [""])))
-
-        super().__init__(json, path, strict, **kwargs)
-
-    def __repr__(self):
-        return f"ComponentManifest({self.id})"
-
-    def isEnabled(self, target: TargetManifest) -> tuple[bool, str]:
-        for k, v in self.enableIf.items():
-            if not k in target.props:
-                logger.log(
-                    f"Component {self.id} disabled by missing {k} in target")
-                return False, f"Missing props '{k}' in target"
-
-            if not target.props[k] in v:
-                vStrs = [f"'{str(x)}'" for x in v]
-                logger.log(
-                    f"Component {self.id} disabled by {k}={target.props[k]} not in {v}")
-                return False, f"Props missmatch for '{k}': Got '{target.props[k]}' but expected {', '.join(vStrs)}"
-
-        return True, ""

osdk/plugins.py

@@ -1,30 +0,0 @@
-import os
-
-import importlib.util as importlib
-from osdk.logger import Logger
-from osdk.shell import readdir
-
-logger = Logger("plugins")
-
-
-def load(path: str):
-    logger.log(f"Loading plugin {path}")
-    spec = importlib.spec_from_file_location("plugin", path)
-
-    if not spec or not spec.loader:
-        logger.error(f"Failed to load plugin {path}")
-        return None
-
-    module = importlib.module_from_spec(spec)
-    spec.loader.exec_module(module)
-
-
-def loadAll():
-    logger.log("Loading plugins...")
-    for files in readdir(os.path.join("meta", "plugins")):
-        if files.endswith(".py"):
-            plugin = load(os.path.join("meta", "plugins", files))
-
-            if plugin:
-                print(f"Loaded plugin {plugin.name}")
-                plugin.init()

osdk/rules.py

@@ -1,48 +0,0 @@
-class Rule:
-    id: str
-    fileIn: list[str]
-    fileOut: list[str]
-    rule: str
-    args: list[str]
-    deps: str | None = None
-
-    def __init__(self, id: str,  fileIn: list[str], fileOut: list[str], rule: str, args: list[str] = [], deps: str | None = None):
-        self.id = id
-        self.fileIn = fileIn
-        self.fileOut = fileOut
-        self.rule = rule
-        self.args = args
-        self.deps = deps
-
-rules: dict[str, Rule] = {
-    "cc": Rule("cc", ["*.c"], ["*.o"], "-c -o $out $in -MD -MF $out.d $flags $cincs $cdefs", ["-std=gnu2x",
-                                                                                          "-Wall",
-                                                                                          "-Wextra",
-                                                                                          "-Werror"], "$out.d"),
-    "cxx": Rule("cxx", ["*.cpp", "*.cc", "*.cxx"], ["*.o"], "-c -o $out $in -MD -MF $out.d $flags $cincs $cdefs", ["-std=gnu++2b",
-                                                                                                           "-Wall",
-                                                                                                           "-Wextra",
-                                                                                                           "-Werror",
-                                                                                                           "-fno-exceptions",
-                                                                                                           "-fno-rtti"], "$out.d"),
-    "as": Rule("as", ["*.s", "*.asm", "*.S"], ["*.o"], "-o $out $in $flags"),
-    "ar": Rule("ar", ["*.o"], ["*.a"], "$flags $out $in"),
-    "ld": Rule("ld", ["*.o", "*.a"], ["*.out"], "-o $out $in $flags"),
-}
-
-
-def append(rule: Rule):
-    rules[rule.id] = rule
-
-
-def byFileIn(fileIn: str) -> Rule | None:
-    for key in rules:
-        rule = rules[key]
-        for ext in rule.fileIn:
-            if fileIn.endswith(ext[1:]):
-                return rule
-    return None
-
-
-def byId(id: str) -> Rule | None:
-    return rules.get(id, None)

osdk/vt100.py

@@ -1,47 +0,0 @@
-BLACK = "\033[0;30m"
-RED = "\033[0;31m"
-GREEN = "\033[0;32m"
-BROWN = "\033[0;33m"
-BLUE = "\033[0;34m"
-PURPLE = "\033[0;35m"
-CYAN = "\033[0;36m"
-LIGHT_GRAY = "\033[0;37m"
-DARK_GRAY = "\033[1;30m"
-LIGHT_RED = "\033[1;31m"
-LIGHT_GREEN = "\033[1;32m"
-YELLOW = "\033[1;33m"
-LIGHT_BLUE = "\033[1;34m"
-LIGHT_PURPLE = "\033[1;35m"
-LIGHT_CYAN = "\033[1;36m"
-LIGHT_WHITE = "\033[1;37m"
-BOLD = "\033[1m"
-FAINT = "\033[2m"
-ITALIC = "\033[3m"
-UNDERLINE = "\033[4m"
-BLINK = "\033[5m"
-NEGATIVE = "\033[7m"
-CROSSED = "\033[9m"
-RESET = "\033[0m"
-
-
-def title(text: str):
-    print(f"{LIGHT_WHITE}{text}{RESET}:")
-
-
-def wordwrap(text: str, width: int = 60, newline="\n") -> str:
-    result = ""
-    curr = 0
-
-    for c in text:
-        if c == " " and curr > width:
-            result += newline
-            curr = 0
-        else:
-            result += c
-            curr += 1
-
-    return result
-
-
-def indent(text: str, indent: int = 4) -> str:
-    return " " * indent + text.replace("\n", "\n" + " " * indent)

pyproject.toml

@@ -0,0 +1,32 @@
+[build-system]
+requires = ["setuptools ~= 68.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "cutekit"
+description = "A build system and package manager for low-level software development"
+authors = [
+    { name = "Keyboard Slayer", email = "joda@cute.engineering" },
+    { name = "Sleepy Monax", email = "nivb@cute.engineering" },
+    { name = "Cyp", email = "cypb@cute.engineering" },
+    { name = "Cute Engineering", email = "contact@cute.engineering" },
+]
+readme = "README.md"
+requires-python = ">=3.11"
+license = { text = "MIT" }
+dynamic = ["version", "dependencies"]
+
+[project.scripts]
+ck = "cutekit:main"
+cutekit = "cutekit:main"
+cute-engineering-cutekit = "cutekit:main"
+
+[tool.setuptools]
+packages = ["cutekit"]
+
+[tool.setuptools.dynamic]
+version = { attr = "cutekit.const.VERSION" }
+dependencies = { file = ["requirements.txt"] }
+
+[tool.setuptools.package-data]
+"cutekit" = ["py.typed"]

requirements.txt

@@ -1,2 +1,3 @@
-requests ~= 2.28.0
+requests ~= 2.31.0
 graphviz ~= 0.20.1
+dataclasses-json ~= 0.6.2

setup.cfg

@@ -1,4 +0,0 @@
-[metadata]
-description-file=README.md
-license_file=LICENSE
-

setup.py

@@ -1,26 +0,0 @@
-from setuptools import setup
-from osdk.const import VERSION
-
-setup(
-    name="osdk",
-    version=VERSION,
-    python_requires='>=3.10',
-    description="Operating System Development Kit",
-    author="Cute Engineering",
-    author_email="contact@cute.engineering",
-    url="https://cute.engineering/",
-    packages=["osdk"],
-    install_requires=[
-        "requests",
-        "graphviz"
-    ],
-    entry_points={
-        "console_scripts": [
-            "osdk = osdk:main",
-        ],
-    },
-    license="MIT",
-    platforms="any",
-    long_description=open("README.md").read(),
-    long_description_content_type="text/markdown",
-)

tests/test_resolver.py

@@ -0,0 +1,90 @@
+from cutekit import model
+
+
+def test_direct_deps():
+    r = model.Registry("")
+    r._append(model.Component("myapp", requires=["mylib"]))
+    r._append(model.Component("mylib"))
+    t = model.Target("host")
+    res = model.Resolver(r, t)
+
+    resolved = res.resolve("myapp")
+    assert resolved.reason is None
+    assert resolved.resolved == ["myapp", "mylib"]
+
+
+def test_indirect_deps():
+    r = model.Registry("")
+    r._append(model.Component("myapp", requires=["mylib"]))
+    r._append(model.Component("mylib", requires=["myembed"]))
+    r._append(model.Component("myimpl", provides=["myembed"]))
+    t = model.Target("host")
+    res = model.Resolver(r, t)
+    assert res.resolve("myapp").resolved == ["myapp", "mylib", "myimpl"]
+
+
+def test_deps_routing():
+    r = model.Registry("")
+    r._append(model.Component("myapp", requires=["mylib"]))
+    r._append(model.Component("mylib", requires=["myembed"]))
+    r._append(model.Component("myimplA", provides=["myembed"]))
+    r._append(model.Component("myimplB", provides=["myembed"]))
+    t = model.Target("host", routing={"myembed": "myimplB"})
+    res = model.Resolver(r, t)
+    assert res.resolve("myapp").resolved == ["myapp", "mylib", "myimplB"]
+
+    t = model.Target("host", routing={"myembed": "myimplA"})
+    res = model.Resolver(r, t)
+    assert res.resolve("myapp").resolved == ["myapp", "mylib", "myimplA"]
+
+    t = model.Target("host", routing={"myembed": "myimplC"})
+    res = model.Resolver(r, t)
+    assert res.resolve("myapp").reason == "No provider for 'myembed'"
+
+
+def test_deps_routing_with_props():
+    r = model.Registry("")
+    r._append(model.Component("myapp", requires=["mylib"]))
+    r._append(model.Component("mylib", requires=["myembed"]))
+    r._append(
+        model.Component("myimplA", provides=["myembed"], enableIf={"myprop": ["a"]})
+    )
+    r._append(
+        model.Component("myimplB", provides=["myembed"], enableIf={"myprop": ["b"]})
+    )
+    t = model.Target("host", routing={"myembed": "myimplB"}, props={"myprop": "b"})
+    res = model.Resolver(r, t)
+    assert res.resolve("myapp").resolved == ["myapp", "mylib", "myimplB"]
+
+    t = model.Target("host", routing={"myembed": "myimplA"}, props={"myprop": "a"})
+    res = model.Resolver(r, t)
+    assert res.resolve("myapp").resolved == ["myapp", "mylib", "myimplA"]
+
+    t = model.Target("host", routing={"myembed": "myimplC"}, props={"myprop": "c"})
+    res = model.Resolver(r, t)
+
+    resolved = res.resolve("myapp")
+    assert resolved.reason == "No provider for 'myembed'"
+
+
+def test_deps_routing_with_props_and_requires():
+    r = model.Registry("")
+    r._append(model.Component("myapp", requires=["mylib"]))
+    r._append(model.Component("mylib", requires=["myembed"]))
+    r._append(
+        model.Component("myimplA", provides=["myembed"], enableIf={"myprop": ["a"]})
+    )
+    r._append(
+        model.Component("myimplB", provides=["myembed"], enableIf={"myprop": ["b"]})
+    )
+    t = model.Target("host", routing={"myembed": "myimplB"}, props={"myprop": "b"})
+    res = model.Resolver(r, t)
+    assert res.resolve("myapp").resolved == ["myapp", "mylib", "myimplB"]
+
+    t = model.Target("host", routing={"myembed": "myimplA"}, props={"myprop": "a"})
+    res = model.Resolver(r, t)
+    assert res.resolve("myapp").resolved == ["myapp", "mylib", "myimplA"]
+
+    t = model.Target("host", routing={"myembed": "myimplC"}, props={"myprop": "c"})
+    res = model.Resolver(r, t)
+    assert res.resolve("myapp").reason == "No provider for 'myembed'"