meta-digi-containers: modify how to generate the DCP

Modify and document the way to generate a DCP by package_id.
Update the source code of the container manager to align with.

Signed-off-by: Isaac Hermida <isaac.hermida@digi.com>
This commit is contained in:
Isaac Hermida 2026-05-27 12:04:46 +02:00
parent 9de4842dc3
commit 7950ac2460
6 changed files with 24 additions and 67 deletions

View File

@ -78,13 +78,11 @@ The script generates exactly one runtime artifact per execution.
Notes: Notes:
- `--output-dir` is optional. If omitted, the script writes the DCP to the current directory. - `--output-dir` is optional. If omitted, the script writes the DCP to the current directory.
- If the input manifest omits `package_id`, the generator derives the final - The input manifest must provide `package_id`, which becomes the stable target
`package_id` from `name` and appends a unique suffix using the `created_at` identifier.
timestamp encoded in base36 milliseconds. - The output file name is always derived from the package ID and
- If the input manifest provides `package_id`, that value is kept unchanged.
- The output file name is always derived from the generated package ID and
manifest fields as: manifest fields as:
- `<generated_package_id>-<runtime>-<device_types[0]>.tar.gz` - `<package_id>-<runtime>-<device_types[0]>.tar.gz`
- For Podman payloads, the final internal payload name is always `payload/image.tar` - For Podman payloads, the final internal payload name is always `payload/image.tar`
- For LXC payloads, the generator requires a Yocto-style `.tar.gz` bundle, validates its layout, - For LXC payloads, the generator requires a Yocto-style `.tar.gz` bundle, validates its layout,
and stores it inside the DCP using the original file name and format without re-packaging it. and stores it inside the DCP using the original file name and format without re-packaging it.
@ -104,14 +102,13 @@ python3 meta-digi-containers/scripts/generate-dcp.py \
This generates a bundle named like: This generates a bundle named like:
- `./flutter-demo-<base36_created_at_ms>-podman-ccmp25-dvk.tar.gz` - `./flutter-demo-podman-ccmp25-dvk.tar.gz`
In those manifests: In those manifests:
- `name` is the stable logical container name stored on the target. - `package_id` is the stable target identifier.
- `friendly_name` is the user-facing label shown by the manager output when available. - `friendly_name` is the user-facing label shown by the manager output when available.
- the final `package_id` is generated automatically from `name` unless the - the final DCP manifest does not include a second container identifier
input manifest provides an explicit `package_id`
- `registration_defaults` is limited to local manager policy such as `autostart`, - `registration_defaults` is limited to local manager policy such as `autostart`,
`monitor`, and `restart`; this release does not generate DRM-specific defaults `monitor`, and `restart`; this release does not generate DRM-specific defaults
@ -182,19 +179,12 @@ Outputs are generated in:
Final outputs: Final outputs:
- `${DCP_NAME}-<base36_created_at_ms>-podman-<device_types[0]>.tar.gz` - `${DCP_NAME}-podman-<device_types[0]>.tar.gz`
- `${DCP_NAME}-<base36_created_at_ms>-lxc-<device_types[0]>.tar.gz` - `${DCP_NAME}-lxc-<device_types[0]>.tar.gz`
Notes: Notes:
- The generator script derives `package_id` from the input `name` only when the - In Yocto builds, `dey-image-dcp` sets `package_id` to `${DCP_NAME}`.
manifest does not provide one explicitly. In that default case it appends a
base36-encoded millisecond timestamp suffix, stores the generated `package_id`
in the final `manifest.json`, and uses it in the DCP file name.
- In Yocto builds, `dey-image-dcp` keeps that unique-name behavior and
removes older DCP artifacts with the same `${DCP_NAME}` prefix before
generating the new one, so the deploy directory does not keep accumulating
previous builds of the same container/runtime.
Intermediate rootfs and OCI outputs are kept available for incremental rebuilds. Intermediate rootfs and OCI outputs are kept available for incremental rebuilds.
The LXC bundle and Podman archive are generated as temporary payloads while The LXC bundle and Podman archive are generated as temporary payloads while
@ -281,7 +271,6 @@ Supported placeholders in LXC config fragments:
The artifact manifest is generated automatically and includes: The artifact manifest is generated automatically and includes:
- `package_id` - `package_id`
- `name` [stable logical container name]
- `friendly_name` [optional user-facing display name] - `friendly_name` [optional user-facing display name]
- `version` - `version`
- `runtime` - `runtime`

View File

@ -1,5 +1,5 @@
{ {
"name": "flutter-demo", "package_id": "flutter-demo",
"friendly_name": "Flutter Demo", "friendly_name": "Flutter Demo",
"version": "1.0", "version": "1.0",
"runtime": "lxc", "runtime": "lxc",

View File

@ -1,5 +1,5 @@
{ {
"name": "flutter-demo", "package_id": "flutter-demo",
"friendly_name": "Flutter Demo", "friendly_name": "Flutter Demo",
"version": "1.0", "version": "1.0",
"runtime": "podman", "runtime": "podman",

View File

@ -15,8 +15,8 @@ SRC_URI = " \
file://cc-containerd.service \ file://cc-containerd.service \
file://cc-containerd-shutdown.service \ file://cc-containerd-shutdown.service \
" "
SRC_URI[archive.md5sum] = "6d84c6f5ec9dc94d542c91001ff5fd36" SRC_URI[archive.md5sum] = "2ae2c3c09e9bf223e7de4ec0994376f3"
SRC_URI[archive.sha256sum] = "ce24c4fde041a69a7646eb9bad4891d2eb91291f3534e71444552d3830247aaa" SRC_URI[archive.sha256sum] = "627d90eb53a48bf978fb6993f661af3dba8e6bf091d295ff481457e4c0cb96a0"
S = "${WORKDIR}/${BP}" S = "${WORKDIR}/${BP}"

View File

@ -15,9 +15,9 @@ do_clean_old_container_artifacts() {
fi fi
for runtime in lxc podman; do for runtime in lxc podman; do
old_artifact_glob="${DEPLOY_DIR_IMAGE}/${DCP_NAME}-*-${runtime}-${primary_device_type}.tar.gz" artifact_path="${DEPLOY_DIR_IMAGE}/${DCP_NAME}-${runtime}-${primary_device_type}.tar.gz"
bbnote "Removing old ${runtime} container artifacts matching ${old_artifact_glob}" bbnote "Removing old ${runtime} container artifact ${artifact_path}"
rm -f ${old_artifact_glob} rm -f "${artifact_path}"
done done
} }
@ -69,7 +69,7 @@ do_image_container_artifacts() {
manifest_path="$2" manifest_path="$2"
create_args="$3" create_args="$3"
build_id="$4" build_id="$4"
NAME="${DCP_NAME}" \ PACKAGE_ID="${DCP_NAME}" \
FRIENDLY_NAME="${CONTAINER_FRIENDLY_NAME}" \ FRIENDLY_NAME="${CONTAINER_FRIENDLY_NAME}" \
VERSION="${CONTAINER_ARTIFACT_VERSION}" \ VERSION="${CONTAINER_ARTIFACT_VERSION}" \
RUNTIME="${runtime}" \ RUNTIME="${runtime}" \
@ -87,7 +87,7 @@ do_image_container_artifacts() {
LABELS_JSON="${CONTAINER_ARTIFACT_LABELS_JSON}" \ LABELS_JSON="${CONTAINER_ARTIFACT_LABELS_JSON}" \
python3 -c 'import json, os, sys; \ python3 -c 'import json, os, sys; \
parse_bool = lambda name: os.environ[name].strip().lower() == "true"; \ parse_bool = lambda name: os.environ[name].strip().lower() == "true"; \
payload = {"name": os.environ["NAME"], "friendly_name": os.environ["FRIENDLY_NAME"], "version": os.environ["VERSION"], "runtime": os.environ["RUNTIME"], "registration_defaults": {"autostart": parse_bool("AUTOSTART"), "monitor": parse_bool("MONITOR"), "restart": {"enabled": parse_bool("RESTART_ENABLED"), "max_retries": int(os.environ["RESTART_MAX_RETRIES"]), "window": int(os.environ["RESTART_WINDOW"]), "retry_delay": int(os.environ["RESTART_RETRY_DELAY"]) } }, "device_types": json.loads(os.environ["DEVICE_TYPES_JSON"]), "firmware_versions": os.environ["FIRMWARE_VERSIONS"], "build_id": os.environ["BUILD_ID"], "description": os.environ["DESCRIPTION"], "labels": json.loads(os.environ["LABELS_JSON"])}; \ payload = {"package_id": os.environ["PACKAGE_ID"], "friendly_name": os.environ["FRIENDLY_NAME"], "version": os.environ["VERSION"], "runtime": os.environ["RUNTIME"], "registration_defaults": {"autostart": parse_bool("AUTOSTART"), "monitor": parse_bool("MONITOR"), "restart": {"enabled": parse_bool("RESTART_ENABLED"), "max_retries": int(os.environ["RESTART_MAX_RETRIES"]), "window": int(os.environ["RESTART_WINDOW"]), "retry_delay": int(os.environ["RESTART_RETRY_DELAY"]) } }, "device_types": json.loads(os.environ["DEVICE_TYPES_JSON"]), "firmware_versions": os.environ["FIRMWARE_VERSIONS"], "build_id": os.environ["BUILD_ID"], "description": os.environ["DESCRIPTION"], "labels": json.loads(os.environ["LABELS_JSON"])}; \
create_args = os.environ["CREATE_ARGS"].strip(); \ create_args = os.environ["CREATE_ARGS"].strip(); \
payload.update({"create_args": create_args} if create_args else {}); \ payload.update({"create_args": create_args} if create_args else {}); \
open(sys.argv[1], "w", encoding="utf-8").write(json.dumps(payload, indent=2) + "\n")' \ open(sys.argv[1], "w", encoding="utf-8").write(json.dumps(payload, indent=2) + "\n")' \

View File

@ -15,32 +15,15 @@ import sys
import tarfile import tarfile
import tempfile import tempfile
BASE36_ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyz"
def fail(message: str) -> "NoReturn": def fail(message: str) -> "NoReturn":
raise SystemExit(f"error: {message}") raise SystemExit(f"error: {message}")
def to_base36(value: int) -> str:
if value < 0:
fail("cannot convert negative values to base36")
if value == 0:
return "0"
result: list[str] = []
while value:
value, remainder = divmod(value, 36)
result.append(BASE36_ALPHABET[remainder])
return "".join(reversed(result))
def format_created_at(timestamp: datetime) -> str: def format_created_at(timestamp: datetime) -> str:
return timestamp.astimezone(timezone.utc).isoformat(timespec="milliseconds").replace("+00:00", "Z") return timestamp.astimezone(timezone.utc).isoformat(timespec="milliseconds").replace("+00:00", "Z")
def build_generated_package_id(base_name: str, *, created_at_ms: int) -> str:
return f"{base_name}-{to_base36(created_at_ms)}"
def load_manifest(path: Path) -> dict: def load_manifest(path: Path) -> dict:
try: try:
data = json.loads(path.read_text(encoding="utf-8")) data = json.loads(path.read_text(encoding="utf-8"))
@ -100,11 +83,7 @@ def validate_labels(value: object) -> dict:
def validate_manifest(data: dict) -> dict: def validate_manifest(data: dict) -> dict:
package_id = data.get("package_id") package_id = validate_string(data, "package_id", path="manifest")
if package_id is not None:
if not isinstance(package_id, str) or not package_id.strip():
fail("invalid manifest: package_id must be a non-empty string")
package_id = package_id.strip()
version = validate_string(data, "version", path="manifest") version = validate_string(data, "version", path="manifest")
runtime = validate_string(data, "runtime", path="manifest") runtime = validate_string(data, "runtime", path="manifest")
if runtime not in {"lxc", "podman"}: if runtime not in {"lxc", "podman"}:
@ -136,7 +115,6 @@ def validate_manifest(data: dict) -> dict:
description = data.get("description", "") description = data.get("description", "")
if description is not None and not isinstance(description, str): if description is not None and not isinstance(description, str):
fail("invalid manifest: description must be a string") fail("invalid manifest: description must be a string")
name = validate_string(data, "name", path="manifest")
friendly_name = data.get("friendly_name") friendly_name = data.get("friendly_name")
if friendly_name is not None: if friendly_name is not None:
if not isinstance(friendly_name, str) or not friendly_name.strip(): if not isinstance(friendly_name, str) or not friendly_name.strip():
@ -145,7 +123,6 @@ def validate_manifest(data: dict) -> dict:
validated = { validated = {
"package_id": package_id, "package_id": package_id,
"name": name,
"friendly_name": friendly_name, "friendly_name": friendly_name,
"version": version, "version": version,
"runtime": runtime, "runtime": runtime,
@ -269,7 +246,6 @@ def build_final_manifest(
) -> dict: ) -> dict:
output = { output = {
"package_id": package_id, "package_id": package_id,
"name": base["name"],
"friendly_name": base["friendly_name"], "friendly_name": base["friendly_name"],
"version": base["version"], "version": base["version"],
"runtime": base["runtime"], "runtime": base["runtime"],
@ -332,18 +308,10 @@ def main() -> int:
fail(f"cannot create output directory {output_dir}: {exc}") fail(f"cannot create output directory {output_dir}: {exc}")
created_at_dt = datetime.now(timezone.utc) created_at_dt = datetime.now(timezone.utc)
created_at_ms = int(created_at_dt.timestamp() * 1000)
created_at = format_created_at(created_at_dt) created_at = format_created_at(created_at_dt)
generated_package_id = ( package_id = manifest["package_id"]
manifest["package_id"]
if manifest["package_id"]
else build_generated_package_id(
manifest["name"],
created_at_ms=created_at_ms,
)
)
output_name = build_output_name( output_name = build_output_name(
package_id=generated_package_id, package_id=package_id,
runtime=manifest["runtime"], runtime=manifest["runtime"],
device_type=manifest["device_types"][0], device_type=manifest["device_types"][0],
) )
@ -375,7 +343,7 @@ def main() -> int:
final_manifest = build_final_manifest( final_manifest = build_final_manifest(
manifest, manifest,
package_id=generated_package_id, package_id=package_id,
artifact_type=artifact_type, artifact_type=artifact_type,
digest=digest, digest=digest,
size_bytes=size_bytes, size_bytes=size_bytes,
@ -389,7 +357,7 @@ def main() -> int:
lambda dst: write_default_readme( lambda dst: write_default_readme(
dst, dst,
manifest, manifest,
package_id=generated_package_id, package_id=package_id,
created_at=created_at, created_at=created_at,
payload_name=payload_name, payload_name=payload_name,
), ),