Bladeren bron

fix(queue): orders and groupings show as finished however their last job ended

They were only re-checked when a print completed or an item was edited,
so a grouping whose last job was cancelled, stopped, deleted, failed at
dispatch or cancelled by a library delete, and an order whose last run
was closed by the stuck-print cleanup, stayed active until a restart.
They are now re-checked on every way a job ends, best-effort after the
change is committed.
maziggy 6 dagen geleden
bovenliggende
commit
a3ba9994fe

+ 1 - 0
CHANGELOG.md

@@ -163,6 +163,7 @@ All notable changes to Bambuddy will be documented in this file.
 - **The frontend build no longer warns about `path` and `crypto` being externalized for the STEP previewer (#2976)** — `occt-import-js`, the Emscripten build behind STEP previews, requires both modules, but only inside its `ENVIRONMENT_IS_NODE` branches; in the browser it loads its `.wasm` from the URL the preview worker passes and draws randomness from `crypto.getRandomValues`. Vite still externalized both and printed two warnings on every build. `vite.config.ts` now drops exactly those two warnings for that one package through `build.rolldownOptions.onLog`, so an externalization anywhere else, or of any other module, still shows.
 
 ### Fixed
+- **Orders and groupings show as finished however their last job ended** — An order or grouping was only re-checked when a print completed, an item was deleted or one was edited. A grouping whose last job was cancelled or stopped, or whose jobs were cancelled because their library file was deleted, an order whose last run was closed by the stuck-print cleanup, and one whose last open job failed before it reached the printer or was deleted, stayed **active** until the next restart. They are now re-checked on every way a job ends.
 - **A job cancelled while it is being sent stays cancelled** — Cancelling a queued job during its upload (the edit dialog even asks you to cancel a job that is being sent) was overwritten when the upload then failed: the job turned into a failure, a failure notification went out, and the next job that needs the previous print to have succeeded was skipped. A failure is now only recorded for a job that is still waiting or printing, and the upload toast of a cancelled job closes instead of spinning. Likewise, a job stopped just as its printer turned busy is no longer put back in the queue, and a stopped job whose nozzle rack pick no longer fit no longer leaves its file on the printer.
 - **"Only if the previous print succeeded" looks at the right printer** — For a job queued for any printer of a model, the check ran before a printer was chosen, against jobs with no printer, so it nearly always let the job go, even onto a printer whose last print had failed; when it did trigger, the job was skipped for good. It now checks the printer it is about to use, passes over one whose last print failed and takes another of the model, and waits if every free one failed last. The queue then shows a **Resume after failure** banner for that model, which clears the gate on its printers, and the "job waiting" notification also goes out when a job that waited for a busy printer comes to wait on such a failure. On PostgreSQL, a job cancelled without an end time (cancelling an order, a pipeline or a kept order run) also counted as the latest print and hid a real failure before it; such cancels now record their end time, and the check no longer ranks a missing one first.
 - **A queued job moved to another printer no longer keeps the old printer's settings** — A job's AMS tray mapping, **Print Anyway** and nozzle choice are worked out for one printer. Moving jobs to another printer (bulk edit, or editing one job) kept them, so a job could print from the slot that held the right material on the old printer but another colour on the new one, and a **Print Anyway** given for the old printer skipped the new printer's filament checks. **Dispatch remaining** of an order also pinned every copy of a job queued for any printer of a model to the printer its latest run happened to land on, with that printer's settings, so the other printers of that model stayed idle. These settings now stay with the old printer and are worked out again for the new one; plate, AMS use, filament overrides and print options are kept. Bulk edit leaves jobs with printer alternatives alone, as editing one does, and a job it moves to a printer no longer also waits for a model.

+ 12 - 0
backend/app/api/routes/print_queue.py

@@ -62,6 +62,7 @@ from backend.app.services.print_batch import (
     load_progress,
     refresh_batch_status,
     refresh_batch_status_for_item,
+    refresh_batches_after,
 )
 from backend.app.services.print_cost_estimate import estimate_queue_source_cost
 from backend.app.services.queue_item_printer import PRINTER_RESOLVED_DEFAULTS, drop_printer_resolved
@@ -2288,6 +2289,7 @@ async def delete_queue_item(
         source_id=item.id,
         status="released",
     )
+    deleted_batch_id = item.batch_id
     if keep_as_cancelled:
         item.status = "cancelled"
         item.completed_at = datetime.now(timezone.utc)
@@ -2298,6 +2300,14 @@ async def delete_queue_item(
     else:
         await db.delete(item)
     await db.commit()
+    if not keep_as_cancelled and deleted_batch_id is not None:
+        # A grouping whose last open job was deleted may be finished now.
+        try:
+            batch = await db.get(PrintBatch, deleted_batch_id)
+            if batch is not None and await refresh_batch_status(db, batch):
+                await db.commit()
+        except Exception as e:
+            logger.warning("Failed to refresh batch %s after a delete: %s", deleted_batch_id, e)
 
     # Stop an in-flight preheat for this item: the dispatch coroutine is
     # parked in a sleep and cannot see the status we just wrote (#2727).
@@ -2471,6 +2481,7 @@ async def cancel_queue_item(
         status="released",
     )
     await db.commit()
+    await refresh_batches_after([item.id])
 
     # Stop an in-flight preheat for this item: the dispatch coroutine is
     # parked in a sleep and cannot see the status we just wrote (#2727).
@@ -2561,6 +2572,7 @@ async def stop_queue_item(
             archive.failure_reason = "Stopped by user (printer was offline)"
 
     await db.commit()
+    await refresh_batches_after([item.id])
 
     logger.info("Stopped printing queue item %s (stop command sent: %s)", item_id, stop_sent)
 

+ 7 - 0
backend/app/services/library_trash.py

@@ -463,6 +463,13 @@ async def release_queue_references(db: AsyncSession, file_ids: list[int]) -> int
                 )
             )
         logger.info("Library delete: cancelled %d queued item(s) whose file was removed", len(rows))
+        # A grouping left with only cancelled jobs is finished. Not on the bare
+        # connection the start-up orphan sweep passes: the batch backfill that
+        # runs at start-up re-checks every batch anyway.
+        if isinstance(db, AsyncSession):
+            from backend.app.services.print_batch import refresh_batches_for_items
+
+            await refresh_batches_for_items(db, [item_id for ids in doomed.values() for item_id in ids])
 
     await db.execute(
         PrintQueueItem.__table__.update()

+ 47 - 0
backend/app/services/print_batch.py

@@ -420,6 +420,53 @@ async def backfill_batch_statuses(db: AsyncSession) -> int:
     return changed
 
 
+async def refresh_batches_for_items(db: AsyncSession, queue_item_ids: list[int]) -> None:
+    """Re-evaluate every batch owning one of *queue_item_ids*, once each.
+
+    For the paths that end items without a print completing (a cancel, a stop,
+    the stranded-item sweep, a library delete): a grouping whose last item was
+    cancelled, or an order whose last run was closed, would otherwise stay
+    active until the next restart. Changes are left for the caller to commit.
+    """
+    if not queue_item_ids:
+        return
+    await db.flush()
+    batch_ids = (
+        (
+            await db.execute(
+                select(PrintQueueItem.batch_id)
+                .where(PrintQueueItem.id.in_(queue_item_ids))
+                .where(PrintQueueItem.batch_id.is_not(None))
+                .distinct()
+            )
+        )
+        .scalars()
+        .all()
+    )
+    for batch_id in batch_ids:
+        batch = await db.get(PrintBatch, batch_id)
+        if batch is not None:
+            await refresh_batch_status(db, batch)
+
+
+async def refresh_batches_after(queue_item_ids: list[int]) -> None:
+    """:func:`refresh_batches_for_items` in its own session, after the caller's commit.
+
+    Best-effort, as on the print-completion path: an order's status must not
+    be the reason a cancel, a stop or a scheduler step fails or rolls back.
+    """
+    if not queue_item_ids:
+        return
+    from backend.app.core import database
+
+    try:
+        async with database.async_session() as db:
+            await refresh_batches_for_items(db, queue_item_ids)
+            await db.commit()
+    except Exception as e:
+        logger.warning("Failed to refresh the batch status of queue items %s: %s", queue_item_ids, e)
+
+
 async def refresh_batch_status_for_item(db: AsyncSession, queue_item_id: int) -> None:
     """Re-evaluate the batch owning *queue_item_id*, if it has one.
 

+ 12 - 0
backend/app/services/print_scheduler.py

@@ -1411,6 +1411,9 @@ class PrintScheduler:
                     )
                 if closed:
                     await db.commit()
+                    from backend.app.services.print_batch import refresh_batches_after
+
+                    await refresh_batches_after([item.id for item in items if item.status != "printing"])
                 for printer_id in list(self._terminal_since):
                     if printer_id not in seen_printers:
                         del self._terminal_since[printer_id]
@@ -4110,6 +4113,9 @@ class PrintScheduler:
         item.completed_at = datetime.now(timezone.utc)
         item.waiting_reason = None
         await db.commit()
+        from backend.app.services.print_batch import refresh_batches_after
+
+        await refresh_batches_after([item.id])
         logger.warning(
             "Queue item %s: no usable AMS mapping on printer %s — %s",
             item.id,
@@ -8224,6 +8230,9 @@ class PrintScheduler:
             logger.info("Queue item %s: not marked failed (%s), it was cancelled meanwhile", item.id, message)
             return False
         await db.refresh(item)
+        from backend.app.services.print_batch import refresh_batches_after
+
+        await refresh_batches_after([item.id])
         return True
 
     @staticmethod
@@ -9671,6 +9680,9 @@ class PrintScheduler:
             # The item is failed: its print user and expected print must not
             # pass the printer's next print off as Bambuddy's.
             scheduler._forget_unstarted_print(queue_item_id, unless_printing=True)
+            from backend.app.services.print_batch import refresh_batches_after
+
+            await refresh_batches_after([queue_item_id])
         elif revert_outcome == "reverted":
             # Another try claims the printer again. Until then no print on it
             # is vouched for as this item's, except a late start of its file.

+ 124 - 0
backend/tests/integration/test_batch_status_after_cancel.py

@@ -0,0 +1,124 @@
+"""An order or grouping reports itself finished however its last job ended."""
+
+from __future__ import annotations
+
+from types import SimpleNamespace
+from unittest.mock import MagicMock, patch
+
+import pytest
+from httpx import AsyncClient
+from sqlalchemy import update
+
+from backend.app.models.print_queue import PrintQueueItem
+from backend.tests.integration.test_print_batch_orders import _create_order, _queue_item
+
+pytestmark = [pytest.mark.asyncio, pytest.mark.integration]
+
+
+async def _batch_status(client: AsyncClient, batch_id: int) -> str:
+    response = await client.get(f"/api/v1/queue/batches/{batch_id}")
+    assert response.status_code == 200, response.text
+    return response.json()["status"]
+
+
+async def test_a_grouping_whose_last_job_is_cancelled_is_cancelled(
+    async_client: AsyncClient, printer_factory, archive_factory
+):
+    printer = await printer_factory()
+    archive = await archive_factory(printer.id)
+    ids = [
+        (await async_client.post("/api/v1/queue/", json={"printer_id": printer.id, "archive_id": archive.id})).json()[
+            "id"
+        ]
+        for _ in range(2)
+    ]
+    grouping = (await async_client.post("/api/v1/queue/batches", json={"name": "G", "item_ids": ids})).json()
+
+    for item_id in ids:
+        assert (await async_client.post(f"/api/v1/queue/{item_id}/cancel")).status_code == 200
+
+    assert await _batch_status(async_client, grouping["id"]) == "cancelled"
+
+
+async def test_an_order_whose_last_run_is_closed_by_the_sweep_is_completed(
+    async_client: AsyncClient, db_session, printer_factory, archive_factory
+):
+    import backend.app.core.database as database
+    from backend.app.services.print_scheduler import PrintScheduler
+
+    printer = await printer_factory()
+    archive = await archive_factory(printer.id)
+    order = await _create_order(async_client, archive.id, [{"plate_id": 1, "quantity_target": 1}])
+    run = await _queue_item(async_client, printer.id, archive.id, order["id"], plate_id=1)
+    await db_session.execute(update(PrintQueueItem).where(PrintQueueItem.id == run["id"]).values(status="printing"))
+    await db_session.commit()
+
+    scheduler = PrintScheduler()
+    scheduler._terminal_since[printer.id] = -10_000.0
+    with (
+        patch("backend.app.services.print_scheduler.async_session", database.async_session),
+        patch(
+            "backend.app.services.print_scheduler.printer_manager.get_status",
+            MagicMock(return_value=SimpleNamespace(state="FINISH", connected=True)),
+        ),
+    ):
+        await scheduler._close_stranded_printing_items()
+
+    assert await _batch_status(async_client, order["id"]) == "completed"
+
+
+async def test_a_grouping_whose_file_is_deleted_is_cancelled(async_client: AsyncClient, db_session, printer_factory):
+    from backend.app.models.library import LibraryFile
+    from backend.app.services.library_trash import release_queue_references
+
+    printer = await printer_factory()
+    file = LibraryFile(filename="part.3mf", file_path="part.3mf", file_type="3mf", file_size=1)
+    db_session.add(file)
+    await db_session.commit()
+    item = PrintQueueItem(printer_id=printer.id, library_file_id=file.id, status="pending", position=1)
+    db_session.add(item)
+    await db_session.commit()
+    grouping = (await async_client.post("/api/v1/queue/batches", json={"name": "G", "item_ids": [item.id]})).json()
+
+    await release_queue_references(db_session, [file.id])
+    await db_session.commit()
+
+    assert await _batch_status(async_client, grouping["id"]) == "cancelled"
+
+
+async def _grouping(client: AsyncClient, printer_id: int, archive_id: int) -> tuple[dict, list[int]]:
+    ids = [
+        (await client.post("/api/v1/queue/", json={"printer_id": printer_id, "archive_id": archive_id})).json()["id"]
+        for _ in range(2)
+    ]
+    grouping = (await client.post("/api/v1/queue/batches", json={"name": "G", "item_ids": ids})).json()
+    return grouping, ids
+
+
+async def test_a_grouping_whose_last_job_fails_at_dispatch_is_finished(
+    async_client: AsyncClient, db_session, printer_factory, archive_factory
+):
+    from backend.app.services.print_scheduler import PrintScheduler
+
+    printer = await printer_factory()
+    archive = await archive_factory(printer.id)
+    grouping, (done, last) = await _grouping(async_client, printer.id, archive.id)
+    await db_session.execute(update(PrintQueueItem).where(PrintQueueItem.id == done).values(status="completed"))
+    await db_session.commit()
+
+    await PrintScheduler()._fail_dispatch(db_session, await db_session.get(PrintQueueItem, last), "Printer not found")
+
+    assert await _batch_status(async_client, grouping["id"]) == "completed"
+
+
+async def test_a_grouping_whose_last_open_job_is_deleted_is_finished(
+    async_client: AsyncClient, printer_factory, archive_factory
+):
+    printer = await printer_factory()
+    archive = await archive_factory(printer.id)
+    grouping, (first, last) = await _grouping(async_client, printer.id, archive.id)
+    assert (await async_client.post(f"/api/v1/queue/{first}/cancel")).status_code == 200
+
+    assert (await async_client.delete(f"/api/v1/queue/{last}")).status_code == 200
+
+    assert await _batch_status(async_client, grouping["id"]) == "cancelled"