Filament Backup: pool spools the printer treats as backups (issue #1824)

- With AMS Filament Backup on, a print is only held when all spools of
  its filament together are short. Which spools count as the same
  filament came from the inventory rows' stored preset, and an RFID
  spool stores "GFA10" while one set up from the picker or a user
  preset stores "GFSA10_02" / "PFUS...": two rolls of the same PLA
  Tough+ never pooled, and Start refused a 194 g print with 142 g and
  1000 g loaded while the printer listed both for Auto Refill.
- The pool now keys each slot on the printer's own tray report
  (tray_info_idx + colour), the rule the firmware and the AMS Backup
  modal use; a slot it does not report falls back to the bound spool
  with the preset normalised to a filament_id. Grams still come from
  the inventory. Same for internal inventory and Spoolman, for Start,
  the scheduler and the print dialog (inventory-remain).
- The demand and the pool share one read of the report, so a status
  push in between cannot split a slot from its own grams.
- Tests: reporter case, picker vs RFID presets, colour / no preset /
  empty tray do not pool, dialog keys, dict vt_tray, malformed
  report, Spoolman parity, single read.

CHANGELOG.md

@@ -216,6 +216,7 @@ All notable changes to Bambuddy will be documented in this file.
 - **The frontend build no longer warns about `path` and `crypto` being externalized for the STEP previewer (#2976)** — `occt-import-js`, the Emscripten build behind STEP previews, requires both modules, but only inside its `ENVIRONMENT_IS_NODE` branches; in the browser it loads its `.wasm` from the URL the preview worker passes and draws randomness from `crypto.getRandomValues`. Vite still externalized both and printed two warnings on every build. `vite.config.ts` now drops exactly those two warnings for that one package through `build.rolldownOptions.onLog`, so an externalization anywhere else, or of any other module, still shows.
 
 ### Fixed
+- **AMS Filament Backup pools spools the printer itself treats as backups (#1824, reported by @lightmaster)** — With Filament Backup on, a print short on its assigned spool is only held when all spools of that filament together are short. Which spools count as the same filament was decided from the spools in the inventory, by their preset as stored: a spool created from its RFID tag stores `GFA10`, one set up from the preset picker `GFSA10_02` or a user preset of its own, so two rolls of the same PLA Tough+ never pooled, and Start reported "Insufficient filament for the assigned spool" for a 194 g print with 142 g in one slot and 1000 g in the other, while the printer listed both for Auto Refill. The pool now follows what the printer reports for each tray, its filament preset and colour, which is what it switches on; the remaining grams still come from the inventory. Internal inventory and Spoolman alike, for the queue's Start, the dispatcher and the print dialog's warning.
 - **One plate of a multi-plate file is measured against that plate (#3375, reported by @sargonas)** — The slicer's estimate on an archive adds all plates of the file up, so printing one plate of a three-plate file showed +200 % on the card, an archive printed more than once showed no badge at all, and statistics only kept time accuracies between 50 % and 200 %. An archive now keeps each plate's estimate from its 3MF (archives made before get theirs in the background after the update), and each print is measured against the plate it printed: the card's badge compares the latest print with its plate's estimate, the archive window shows each print's duration and filament next to its plate's estimate, the statistics' time accuracy goes by each print's plate, and the print start and queue job started notifications give the plate's time (the printer's own remaining time when the plate of a multi-plate file is not known). Moving a queued job to another plate now also updates the time shortest-job-first orders it by. A print of a multi-plate file whose plate is not known gets no estimate rather than the whole file's. Wiki updated.
 - **Git backup: prints come back with their archives** — A Git backup held archives only, so restoring onto a rebuilt instance gave archives without their prints: statistics and the Print Log came back empty, and each print's printer, plate, outcome, cost and energy was lost. The backup now also writes each print (`archives/print_runs.json`) and which archive was merged into which (`archives/aliases.json`), and **Print archives** restores both: each print is attached to its restored archive, which then shows its latest print's outcome. An archive printed again since the backup is recognised by when it was archived and its prints' start times instead of being added a second time, and an archive merged into another, before or after the backup, is not restored as one of its own, with its spool usage going to the archive it went into. A print running when the backup was made comes back cancelled, like its archive. The one-tap confirm link, photos, timelapses and wallet links are not in the backup. Older backups restore as before. Wiki updated.
 - **Orca Cloud presets with quoted string lists** - Orca Cloud returns lists like compatible_printers as one INI-quoted string, so its processes were hidden for every printer, and a process made for a user printer preset was refused by the slicer. Bambuddy now reads these lists the way OrcaSlicer does, in the preset list and on the slice path. Reported in #3380, fixed by @pleasantone in #3381.

backend/app/services/filament_deficit.py

@@ -44,6 +44,7 @@ from backend.app.models.print_queue import PrintQueueItem
 from backend.app.models.spool_assignment import SpoolAssignment
 from backend.app.models.spoolman_slot_assignment import SpoolmanSlotAssignment
 from backend.app.services.filament_requirements import extract_filament_requirements
+from backend.app.utils.filament_ids import normalize_slicer_filament
 
 logger = logging.getLogger(__name__)
 
@@ -217,8 +218,12 @@ def _material_identity_internal(spool) -> str:
     """
     preset = (spool.slicer_filament or "").strip() if spool else ""
     if preset:
+        # The field holds a filament_id from an RFID scan ("GFA10") or a
+        # setting_id from the preset picker ("GFSA10_02"); both name the same
+        # filament, so compare the filament_id (#1824).
+        filament_id = normalize_slicer_filament(preset)[0] or preset
         color = _normalize_color_for_id(spool.rgba if spool else None)
-        return f"preset:{preset}|color:{color}"
+        return f"preset:{filament_id}|color:{color}"
     # Unique-per-spool key prevents grouping. Use the spool's primary key so
     # the same spool always resolves to the same key within a request.
     spool_id = getattr(spool, "id", None) if spool else None
@@ -255,6 +260,62 @@ def _material_identity_spoolman(spool: dict | None) -> str:
     return f"unmatched:{spool_id}"
 
 
+def _live_tray_keys(printer_id: int) -> dict[int, str]:
+    """Pooling key per ``global_tray_id``, from what the printer reports per tray.
+
+    AMS Filament Backup is the firmware's decision, and it makes it on the
+    tray's own ``tray_info_idx`` and colour. The spool rows bound to two slots
+    can disagree about that while the printer does not: an RFID-created spool
+    stores "GFA10", one set up from a user preset "PFUS..." (#1824). So a slot
+    the printer reports is keyed on the report, in the same format
+    ``_material_identity_internal`` uses; a tray with nothing in it or no
+    preset gets a key of its own, so it never pairs. Slots missing from the
+    report fall back to the bound spool.
+    """
+    try:
+        from backend.app.services.printer_manager import printer_manager
+    except ImportError:
+        return {}
+    state = printer_manager.get_status(printer_id)
+    raw = getattr(state, "raw_data", None) if state is not None else None
+    if not isinstance(raw, dict):
+        return {}
+
+    keys: dict[int, str] = {}
+
+    def _add(global_tray_id: int, tray: dict) -> None:
+        preset = tray.get("tray_info_idx")
+        preset = preset.strip() if isinstance(preset, str) else ""
+        if tray.get("tray_type") and preset:
+            keys[global_tray_id] = f"preset:{preset}|color:{_normalize_color_for_id(tray.get('tray_color'))}"
+        else:
+            keys[global_tray_id] = f"unmatched:tray:{global_tray_id}"
+
+    # Free-form MQTT payload on the dispatch path: a malformed unit or tray
+    # costs that slot its live key, never the check.
+    for unit in raw.get("ams") or []:
+        if not isinstance(unit, dict):
+            continue
+        for tray in unit.get("tray") or []:
+            if not isinstance(tray, dict):
+                continue
+            try:
+                ams_id, tray_id = int(unit.get("id", "")), int(tray.get("id", ""))
+            except (TypeError, ValueError):
+                continue
+            _add(_ams_key_to_global(ams_id, tray_id), tray)
+    vt_trays = raw.get("vt_tray") or []
+    for tray in [vt_trays] if isinstance(vt_trays, dict) else vt_trays:
+        if not isinstance(tray, dict):
+            continue
+        try:
+            global_tray_id = int(tray.get("id", 254))
+        except (TypeError, ValueError):
+            continue
+        _add(global_tray_id, tray)
+    return keys
+
+
 def _extruder_side_for_ams(
     ams_id: int,
     ams_extruder_map: dict[str, int],
@@ -433,7 +494,7 @@ class SlotMaterial:
     global_tray_id: int
     # Opaque grouping key. Two slots pool for AMS Filament Backup only when
     # their keys AND extruder sides match. Never parse it — the format is an
-    # implementation detail of ``_material_identity_*``.
+    # implementation detail of ``_live_tray_keys`` / ``_material_identity_*``.
     material_key: str
     remaining_grams: float
     extruder: int
@@ -458,7 +519,12 @@ class SlotMaterial:
         }
 
 
-async def build_slot_materials(db: AsyncSession, printer_id: int) -> list[SlotMaterial]:
+async def build_slot_materials(
+    db: AsyncSession,
+    printer_id: int,
+    *,
+    live_keys: dict[int, str] | None = None,
+) -> list[SlotMaterial]:
     """Every inventory-bound slot on ``printer_id``, with identity and remaining grams.
 
     Mode-agnostic: internal inventory resolves via ``SpoolAssignment`` joined to
@@ -474,6 +540,10 @@ async def build_slot_materials(db: AsyncSession, printer_id: int) -> list[SlotMa
     warning and the dispatcher's 409 can't disagree about what backs what up.
     """
     _, ams_extruder_map, is_dual = await _get_printer_backup_context(printer_id)
+    # ``live_keys``: a caller that has keyed its own slots passes the same read,
+    # so a status push in between cannot put them in different pools.
+    if live_keys is None:
+        live_keys = _live_tray_keys(printer_id)
     materials: list[SlotMaterial] = []
 
     def _append(
@@ -484,12 +554,13 @@ async def build_slot_materials(db: AsyncSession, printer_id: int) -> list[SlotMa
         spool: SlotSpoolIdentity | None = None,
         cost_per_kg: float | None = None,
     ) -> None:
+        global_tray_id = _ams_key_to_global(ams_id, tray_id)
         materials.append(
             SlotMaterial(
                 ams_id=ams_id,
                 tray_id=tray_id,
-                global_tray_id=_ams_key_to_global(ams_id, tray_id),
-                material_key=material_key,
+                global_tray_id=global_tray_id,
+                material_key=live_keys.get(global_tray_id, material_key),
                 remaining_grams=remaining,
                 extruder=_extruder_side_for_ams(ams_id, ams_extruder_map, is_dual),
                 spool=spool,
@@ -658,10 +729,12 @@ async def compute_deficit_for_queue_item(
 
     spoolman_mode = await _is_spoolman_mode(db)
     backup_on, ams_extruder_map, is_dual = await _get_printer_backup_context(printer_id)
+    live_keys = _live_tray_keys(printer_id) if backup_on else {}
 
     # ------------------------------------------------------------------ phase 1
     # Resolve each requirement to (ams_id, tray_id, identity, remaining_grams).
-    # Slot identity is the identity of the spool *assigned to that slot*. A
+    # Slot identity is what the printer reports for the tray, else the spool
+    # *assigned to that slot* (see ``_live_tray_keys``). A
     # ``None`` remaining means "couldn't determine" — treated as "no deficit"
     # below (preserved from pre-#1762 behaviour for non-backup paths too).
     @dataclass
@@ -786,7 +859,9 @@ async def compute_deficit_for_queue_item(
                 tray_id=tray_id,
                 global_tray_id=global_tray_id,
                 required=float(used_grams),
-                identity=identity,
+                # Must key the same way as ``build_slot_materials``, or the
+                # slot's demand lands in a pool its own grams are not in.
+                identity=live_keys.get(global_tray_id, identity),
                 remaining=remaining,
                 filament_type=str(req.get("type", "")),
                 extruder=_extruder_side_for_ams(ams_id, ams_extruder_map, is_dual),
@@ -821,7 +896,7 @@ async def compute_deficit_for_queue_item(
     pool_by_key: dict[tuple[str, int], float] = defaultdict(float)
     required_by_key: dict[tuple[str, int], float] = defaultdict(float)
 
-    for slot in await build_slot_materials(db, printer_id):
+    for slot in await build_slot_materials(db, printer_id, live_keys=live_keys):
         pool_by_key[(slot.material_key, slot.extruder)] += slot.remaining_grams
 
     for row in resolved:

backend/tests/unit/services/test_filament_deficit.py

@@ -493,14 +493,19 @@ class TestFilamentDeficitBackupAware:
         backup_on: bool,
         ams_extruder_map: dict | None = None,
         model: str | None = None,
+        raw_data: dict | None = None,
     ):
-        """Patch ``printer_manager.get_status`` + ``get_model`` for the test."""
+        """Patch ``printer_manager.get_status`` + ``get_model`` for the test.
+
+        ``raw_data`` is the printer's own tray report; without it every slot is
+        keyed on its bound spool."""
         from types import SimpleNamespace
         from unittest.mock import patch as _patch
 
         fake_state = SimpleNamespace(
             ams_filament_backup=backup_on if backup_on is not None else None,
             ams_extruder_map=ams_extruder_map or {},
+            raw_data=raw_data,
         )
 
         return [
@@ -1352,3 +1357,178 @@ class TestSlotSpoolIdentity:
         assert slots[0].remaining_grams == 500.0
         assert slots[0].spool is None
         assert slots[0].to_dict()["spool"] is None
+
+
+def _tray(tray_id: int, *, preset: str = "GFA10", color: str = "AFB1AEFF", tray_type: str = "PLA") -> dict:
+    return {"id": str(tray_id), "tray_type": tray_type, "tray_info_idx": preset, "tray_color": color}
+
+
+class TestBackupPoolKeys:
+    """#1824: two PLA Tough+ Gray spools, B3 with 142 g and A4 with 1000 g, a
+    194 g print, AMS Filament Backup ON — and Start refused with "Insufficient
+    filament for the assigned spool" while the printer itself listed both for
+    Auto Refill. A4 was created from its RFID tag ("GFA10"); B3 had been set up
+    another way, and the pool compared the two spool rows' preset text."""
+
+    REPORTER_RAW = {"ams": [{"id": "0", "tray": [_tray(3)]}, {"id": "1", "tray": [_tray(2)]}]}
+
+    async def _reporter_setup(self, db_session, printer_factory, tmp_path, *, b3_preset: str):
+        printer = await printer_factory(model="X1C")
+        archive = await _setup_archive_3mf(
+            db_session,
+            tmp_path,
+            [{"id": "1", "type": "PLA", "color": "#AFB1AE", "used_g": "194.0"}],
+        )
+        b3 = await _spool(db_session, label_weight=1000, weight_used=857.52, color="#AFB1AE", slicer_filament=b3_preset)
+        a4 = await _spool(db_session, label_weight=1000, weight_used=0.0, color="AFB1AEFF", slicer_filament="GFA10")
+        await _assign(db_session, printer_id=printer.id, spool_id=b3.id, ams_id=1, tray_id=2)
+        await _assign(db_session, printer_id=printer.id, spool_id=a4.id, ams_id=0, tray_id=3)
+        # Prefer-lowest picked B3 (global tray 6), as in the reporter's log.
+        item = await _queue_item(db_session, printer_id=printer.id, archive=archive, ams_mapping=[6])
+        return printer, item
+
+    async def _deficit(self, db_session, printer, item, raw_data):
+        patches = TestFilamentDeficitBackupAware._patch_status(
+            printer_id=printer.id, backup_on=True, model="X1C", raw_data=raw_data
+        )
+        with patch("backend.app.services.filament_deficit.app_settings.base_dir", Path("/")):
+            for p in patches:
+                p.start()
+            try:
+                return await compute_deficit_for_queue_item(db_session, item)
+            finally:
+                for p in patches:
+                    p.stop()
+
+    async def _slots(self, db_session, printer, raw_data, model="X1C"):
+        from backend.app.services.filament_deficit import build_slot_materials
+
+        patches = TestFilamentDeficitBackupAware._patch_status(
+            printer_id=printer.id, backup_on=True, model=model, raw_data=raw_data
+        )
+        for p in patches:
+            p.start()
+        try:
+            return {s.global_tray_id: s for s in await build_slot_materials(db_session, printer.id)}
+        finally:
+            for p in patches:
+                p.stop()
+
+    @pytest.mark.asyncio
+    async def test_pools_on_what_the_printer_reports(self, db_session, printer_factory, tmp_path):
+        """B3 bound to a user preset, A4 to the RFID id: the printer reports both
+        trays as GFA10 in the same grey, so they pool and 1142 g covers 194 g."""
+        printer, item = await self._reporter_setup(
+            db_session, printer_factory, tmp_path, b3_preset="PFUSaae244ee3ae94c"
+        )
+        assert await self._deficit(db_session, printer, item, self.REPORTER_RAW) == []
+
+    @pytest.mark.asyncio
+    async def test_picker_and_rfid_presets_pool_without_a_report(self, db_session, printer_factory, tmp_path):
+        """Where the printer reports no trays, the bound spools decide: the picker's
+        setting_id and the tag's filament_id name the same filament."""
+        printer, item = await self._reporter_setup(db_session, printer_factory, tmp_path, b3_preset="GFSA10_02")
+        assert await self._deficit(db_session, printer, item, None) == []
+
+    @pytest.mark.asyncio
+    async def test_reported_colours_that_differ_do_not_pool(self, db_session, printer_factory, tmp_path):
+        """The printer is the judge both ways: spool rows that agree do not pool
+        trays it reports in different colours."""
+        printer, item = await self._reporter_setup(db_session, printer_factory, tmp_path, b3_preset="GFA10")
+        raw = {"ams": [{"id": "0", "tray": [_tray(3, color="000000FF")]}, {"id": "1", "tray": [_tray(2)]}]}
+        deficit = await self._deficit(db_session, printer, item, raw)
+        assert [(d.ams_id, d.tray_id) for d in deficit] == [(1, 2)]
+
+    @pytest.mark.asyncio
+    async def test_reported_tray_without_a_preset_does_not_pool(self, db_session, printer_factory, tmp_path):
+        printer, item = await self._reporter_setup(db_session, printer_factory, tmp_path, b3_preset="GFA10")
+        raw = {"ams": [{"id": "0", "tray": [_tray(3, preset="")]}, {"id": "1", "tray": [_tray(2, preset="")]}]}
+        deficit = await self._deficit(db_session, printer, item, raw)
+        assert [(d.ams_id, d.tray_id) for d in deficit] == [(1, 2)]
+
+    @pytest.mark.asyncio
+    async def test_reported_empty_tray_does_not_pool(self, db_session, printer_factory, tmp_path):
+        """A spool still bound to a slot the printer reports empty is not loaded."""
+        printer, item = await self._reporter_setup(db_session, printer_factory, tmp_path, b3_preset="GFA10")
+        raw = {"ams": [{"id": "0", "tray": [_tray(3, tray_type="")]}, {"id": "1", "tray": [_tray(2)]}]}
+        deficit = await self._deficit(db_session, printer, item, raw)
+        assert [(d.ams_id, d.tray_id) for d in deficit] == [(1, 2)]
+
+    @pytest.mark.asyncio
+    async def test_print_dialog_pool_uses_the_same_keys(self, db_session, printer_factory, tmp_path):
+        """``inventory-remain`` hands the Print dialog these keys; they must pool
+        the same two slots the dispatcher does."""
+        printer, _ = await self._reporter_setup(db_session, printer_factory, tmp_path, b3_preset="PFUSaae244ee3ae94c")
+        slots = await self._slots(db_session, printer, self.REPORTER_RAW)
+        assert slots[3].material_key == slots[6].material_key
+
+    @pytest.mark.asyncio
+    async def test_external_tray_reported_as_a_dict(self, db_session, printer_factory):
+        """Some firmware sends ``vt_tray`` as one object rather than a list."""
+        printer = await printer_factory(model="X1C")
+        ext = await _spool(db_session, label_weight=1000, weight_used=0.0, color="#AFB1AE", slicer_filament="PFUS1")
+        ams = await _spool(db_session, label_weight=1000, weight_used=0.0, color="#AFB1AE", slicer_filament="GFA10")
+        await _assign(db_session, printer_id=printer.id, spool_id=ext.id, ams_id=255, tray_id=0)
+        await _assign(db_session, printer_id=printer.id, spool_id=ams.id, ams_id=0, tray_id=0)
+        raw = {"ams": [{"id": "0", "tray": [_tray(0)]}], "vt_tray": {"id": "254", **_tray(254)}}
+        slots = await self._slots(db_session, printer, raw)
+        assert slots[254].material_key == slots[0].material_key
+
+    @pytest.mark.asyncio
+    async def test_malformed_report_falls_back_to_the_spools(self, db_session, printer_factory):
+        printer = await printer_factory(model="X1C")
+        # A colour no tray reports, so a key read off the report would show.
+        a = await _spool(db_session, label_weight=1000, weight_used=0.0, color="#123456", slicer_filament="GFA10")
+        b = await _spool(db_session, label_weight=1000, weight_used=0.0, color="#123456", slicer_filament="PFUS1")
+        await _assign(db_session, printer_id=printer.id, spool_id=a.id, ams_id=0, tray_id=0)
+        await _assign(db_session, printer_id=printer.id, spool_id=b.id, ams_id=0, tray_id=1)
+        # Each unusable entry would key tray 0 or 1 alike if it were read.
+        raw = {
+            "ams": [
+                "junk",
+                {"id": None, "tray": [_tray(0), _tray(1)]},
+                {"id": "0", "tray": [None, {**_tray(0), "id": "x"}]},
+            ]
+        }
+        slots = await self._slots(db_session, printer, raw)
+        assert slots[0].material_key == "preset:GFA10|color:123456"
+        assert slots[1].material_key == "preset:PFUS1|color:123456"
+
+    @pytest.mark.asyncio
+    async def test_spoolman_mode_pools_on_what_the_printer_reports(self, db_session, printer_factory):
+        """Parity: two Spoolman catalogue entries for the same filament pool when
+        the printer reports the trays alike."""
+        from unittest.mock import AsyncMock
+
+        from backend.app.models.spoolman_slot_assignment import SpoolmanSlotAssignment
+
+        printer = await printer_factory(model="X1C")
+        db_session.add(Settings(key="spoolman_enabled", value="true"))
+        for (ams_id, tray_id), spool_id in (((1, 2), 68), ((0, 3), 69)):
+            db_session.add(
+                SpoolmanSlotAssignment(
+                    printer_id=printer.id, ams_id=ams_id, tray_id=tray_id, spoolman_spool_id=spool_id
+                )
+            )
+        await db_session.commit()
+        spools = {
+            68: {"id": 68, "remaining_weight": 142.0, "filament": {"id": 7, "color_hex": "AFB1AE"}},
+            69: {"id": 69, "remaining_weight": 1000.0, "filament": {"id": 8, "color_hex": "AFB1AE"}},
+        }
+        client = AsyncMock()
+        client.get_spool = AsyncMock(side_effect=lambda sid: spools[sid])
+        with patch("backend.app.services.spoolman.get_spoolman_client", AsyncMock(return_value=client)):
+            slots = await self._slots(db_session, printer, self.REPORTER_RAW)
+            without_report = await self._slots(db_session, printer, None)
+        assert slots[3].material_key == slots[6].material_key
+        # Without a report the catalogue entries decide, as before.
+        assert without_report[3].material_key != without_report[6].material_key
+
+    @pytest.mark.asyncio
+    async def test_demand_and_pool_share_one_read_of_the_report(self, db_session, printer_factory, tmp_path):
+        """A status push between keying the demand and building the pool must not
+        put a slot's demand in a pool its own grams are missing from."""
+        printer, item = await self._reporter_setup(db_session, printer_factory, tmp_path, b3_preset="GFA10")
+        reads = iter([{6: "first", 3: "first"}, {6: "second", 3: "second"}])
+        with patch("backend.app.services.filament_deficit._live_tray_keys", lambda pid: next(reads)):
+            assert await self._deficit(db_session, printer, item, None) == []