Просмотр исходного кода

Spools keep their ids when the inventory moves by CSV (#3520)

- The CSV export now writes each spool's id; printed labels carry it in
  their QR code.
- Import keeps an id that this install never handed out (above the
  highest id ever given out, from sqlite_sequence or the PG sequence);
  an id in use, a deleted spool's or a duplicate gets a new one, with
  one preview warning. A bad id is a row error.
- Rows keeping their id are inserted first; on PostgreSQL the spool
  sequence is then raised past them.
- The preview shows the id each spool keeps.
- Tests on SQLite and PostgreSQL, frontend test for the column.
maziggy 1 неделя назад
Родитель
Сommit
9bf3478278

+ 1 - 0
CHANGELOG.md

@@ -27,6 +27,7 @@ All notable changes to Bambuddy will be documented in this file.
 - **"Any model" jobs go to the printer with the fewest print hours (#3480, sponsor vote Q4 2026)** — When several printers of a model can take a job, the queue used to pick the first one in its list, so on a farm the same printer got most of the work. It now picks the one with the fewest print hours (the Maintenance counter, including baseline hours entered there), so jobs and wear spread across the farm. Preferred colours still come first, and the queue powers on the least used printer too when it has to switch one on with an Auto Power On plug.
 
 ### Fixed
+- **Spools keep their ids when the inventory moves by CSV (#3520, reported by @hurr1k4ne)** — The CSV export had no id column, so importing it on another instance numbered the spools anew, in the order of the file, and every printed label, whose QR code carries the spool's id, found the wrong spool or none. The export now writes each spool's id, and the import keeps it when that instance has never used it, which on a fresh instance is every id. An id already used there, also by a deleted spool, or named twice in the file gets a new one, and the preview says how many. The preview shows the id each spool keeps. A CSV without the column imports as before.
 - **A spool with a custom cloud preset gets it from a SpoolBuddy client too (#3519, reported by @pleasantone)** — With login on, each user signs in to Bambu Cloud separately. Assigning a spool from an API key, as a SpoolBuddy client does, had no signed-in user, so its custom preset (PFUS or PFCN) could not be looked up and the slot got Generic PLA, while the same assign from the web UI got the preset's own filament. Configuring a pre-assigned slot when the spool is inserted had the same problem, even for assignments made in the web UI. Both now use a user's sign-in: the API key's owner, and whoever made the assignment. A custom preset that still can't be looked up is now named in the log.
 - **Closing a camera view no longer leaves a CPU core at 100% (#3515, reported by @BrainDeLook)** — A viewer leaving while its stream was reading from the SQLite database could leave that connection's close waiting forever, on a bug in aiosqlite (omnilib/aiosqlite#403). Bambuddy then kept one CPU core fully busy, even with every viewer closed, and other pages such as Projects and Spool Inventory became very slow until a restart. Bambuddy now works around the aiosqlite bug, for every database connection, not only the camera's, and the camera stream's short reads (the chamber light settings and the periodic access check) finish before a leaving viewer ends the stream. PostgreSQL was not affected.
 - **The external spool keeps its filament after Configure Slot on an A1 or A1 mini (#3513, reported by @SaadShaya7)** — These printers report a change to the external spool with only what changed, often just the colour. Bambuddy stored that as the whole slot, so after a colour or preset change the spool showed as empty, with no filament type, until the printer's next full update. The check that the printer took the new preset also could not find the slot and reported it as not confirmed. A partial update now changes only what it carries, and the check confirms the new preset as soon as the printer reports it.

+ 54 - 24
backend/app/api/routes/inventory.py

@@ -7,7 +7,7 @@ import httpx
 from fastapi import APIRouter, Depends, File, HTTPException, Query, UploadFile
 from fastapi.responses import Response, StreamingResponse
 from pydantic import BaseModel, Field, field_validator
-from sqlalchemy import case, delete, func, select
+from sqlalchemy import case, delete, func, select, text
 from sqlalchemy.exc import IntegrityError
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy.orm import selectinload
@@ -22,7 +22,7 @@ from backend.app.core.auth import (
     require_auth_if_enabled,
 )
 from backend.app.core.catalog_defaults import DEFAULT_COLOR_CATALOG, DEFAULT_SPOOL_CATALOG
-from backend.app.core.database import get_db
+from backend.app.core.database import get_db, is_sqlite
 from backend.app.core.permissions import Permission
 from backend.app.core.printer_scope import PrinterScope
 from backend.app.core.websocket import ws_manager
@@ -91,6 +91,7 @@ from backend.app.services.spool_csv import (
     MAX_CSV_IMPORT_BYTES,
     ImportPreview,
     ImportResult,
+    ImportRowResult,
     parse_and_validate,
     serialize,
 )
@@ -1494,6 +1495,22 @@ async def export_spools_csv(
     )
 
 
+async def _continue_spool_ids_after(db: AsyncSession, highest: int) -> None:
+    """Make the next new spool's id follow the ids an import kept (#3520).
+
+    The kept rows are flushed first. SQLite's AUTOINCREMENT then already
+    continues after them; a PostgreSQL sequence knows nothing of ids given
+    explicitly and would hand out one of them next. Only ever raised: every
+    kept id is above what the sequence has given out (``_decide_ids``).
+    """
+    await db.flush()
+    if is_sqlite():
+        return
+    sequence = (await db.execute(text("SELECT pg_get_serial_sequence('spool', 'id')"))).scalar()
+    if sequence:
+        await db.execute(text("SELECT setval(:s, :v)"), {"s": sequence, "v": highest})
+
+
 @router.post("/spools/import", response_model=ImportPreview | ImportResult)
 async def import_spools_csv(
     file: UploadFile = File(...),
@@ -1535,29 +1552,42 @@ async def import_spools_csv(
     if dry_run:
         return preview
 
-    created = 0
-    for row in preview.rows:
-        if row.status == "valid" and row.spool is not None:
-            # Deliberately no material-number inheritance here (#2870), unlike
-            # the other create paths: the file is authoritative. A CSV that
-            # leaves the column blank is stating "no number", not asking for
-            # one to be guessed from whatever else is in the inventory.
-            # The file has no lock column: a used weight in it is locked.
-            spool = Spool(**_lock_entered_weight(dict(row.spool), set()))
-            db.add(spool)
-            # Supplier assignments resolved by name during parsing (#2988).
-            # Flush first so the spool has an id to hang the links on.
-            if row.supplier_ids:
-                await db.flush()
-                for supplier_id in row.supplier_ids:
-                    db.add(
-                        SpoolSupplier(
-                            spool_id=spool.id,
-                            supplier_id=supplier_id,
-                            is_purchase_source=supplier_id == row.purchase_supplier_id,
-                        )
+    async def add_spool(row: ImportRowResult) -> None:
+        # Deliberately no material-number inheritance here (#2870), unlike
+        # the other create paths: the file is authoritative. A CSV that
+        # leaves the column blank is stating "no number", not asking for
+        # one to be guessed from whatever else is in the inventory.
+        # The file has no lock column: a used weight in it is locked.
+        spool = Spool(**_lock_entered_weight(dict(row.spool or {}), set()))
+        if row.keeps_id is not None:
+            spool.id = row.keeps_id
+        db.add(spool)
+        # Supplier assignments resolved by name during parsing (#2988).
+        # Flush first so the spool has an id to hang the links on.
+        if row.supplier_ids:
+            await db.flush()
+            for supplier_id in row.supplier_ids:
+                db.add(
+                    SpoolSupplier(
+                        spool_id=spool.id,
+                        supplier_id=supplier_id,
+                        is_purchase_source=supplier_id == row.purchase_supplier_id,
                     )
-            created += 1
+                )
+
+    valid_rows = [row for row in preview.rows if row.status == "valid" and row.spool is not None]
+    # Rows keeping their id go in first (#3520): a row without one takes the
+    # next free id, which could be one a later row in the file still needs.
+    kept_ids = [row.keeps_id for row in valid_rows if row.keeps_id is not None]
+    for row in valid_rows:
+        if row.keeps_id is not None:
+            await add_spool(row)
+    if kept_ids:
+        await _continue_spool_ids_after(db, max(kept_ids))
+    for row in valid_rows:
+        if row.keeps_id is None:
+            await add_spool(row)
+    created = len(valid_rows)
 
     if created:
         await db.commit()

+ 66 - 1
backend/app/services/spool_csv.py

@@ -17,9 +17,10 @@ import io
 from datetime import datetime, timezone
 
 from pydantic import BaseModel, ValidationError
-from sqlalchemy import select
+from sqlalchemy import func, select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from backend.app.core.database import is_sqlite
 from backend.app.models.color_catalog import ColorCatalogEntry
 from backend.app.models.spool import Spool
 from backend.app.models.supplier import Supplier, supplier_name_key
@@ -38,6 +39,11 @@ from backend.app.schemas.spool import SpoolCreate
 # `material_number` (#2870) are SpoolCreate fields included so a round-trip
 # preserves them (they'd otherwise be lost).
 CSV_COLUMNS = [
+    # The spool's id (#3520): a printed label's QR code carries it, so moving
+    # the inventory to another install must not renumber the spools. Import
+    # keeps it when this install never handed it out (see `_decide_ids`), and
+    # gives the spool a new one otherwise; an empty cell is a new spool.
+    "id",
     "material",
     "brand",
     "subtype",
@@ -102,6 +108,8 @@ class ImportRowResult(BaseModel):
 
     row_number: int  # 1-based data row (header is not counted)
     status: str  # "valid" | "error" | "skipped"
+    # The id the spool is created with (#3520); None gets the next free one.
+    keeps_id: int | None = None
     reason: str | None = None
     material: str | None = None
     brand: str | None = None
@@ -390,6 +398,16 @@ async def parse_and_validate(raw_bytes: bytes, db: AsyncSession) -> ImportPrevie
 
         row_error: str | None = None
 
+        requested_id: int | None = None
+        id_cell = cell(raw_row, "id")
+        if id_cell:
+            try:
+                requested_id = int(id_cell)
+            except ValueError:
+                requested_id = 0
+            if requested_id < 1:
+                row_error = f"id must be a positive whole number (got '{id_cell}')"
+
         # Plain text passthrough columns.
         for field in (
             "subtype",
@@ -543,6 +561,7 @@ async def parse_and_validate(raw_bytes: bytes, db: AsyncSession) -> ImportPrevie
                 resolved_color=resolved_color,
                 cross_material_color=cross_material_color,
                 duplicate_of_existing=_spool_key(material, brand, color_name) in existing_keys,
+                keeps_id=requested_id,
                 spool=spool_data,
                 supplier_ids=supplier_ids,
                 purchase_supplier_id=purchase_supplier_id,
@@ -550,6 +569,8 @@ async def parse_and_validate(raw_bytes: bytes, db: AsyncSession) -> ImportPrevie
         )
         valid += 1
 
+    await _decide_ids(db, rows, warnings)
+
     return ImportPreview(
         columns=CSV_COLUMNS,
         total=valid + error + skipped,
@@ -561,6 +582,50 @@ async def parse_and_validate(raw_bytes: bytes, db: AsyncSession) -> ImportPrevie
     )
 
 
+async def highest_spool_id_given_out(db: AsyncSession) -> int:
+    """The highest spool id this install has ever handed out, deleted ones included.
+
+    Spool ids are never reused (old print logs and usage records may still
+    name one), so only an id above this is free to give an imported spool.
+    """
+    highest = (await db.execute(select(func.max(Spool.id)))).scalar() or 0
+    if is_sqlite():
+        seq = (await db.execute(text("SELECT seq FROM sqlite_sequence WHERE name = 'spool'"))).scalar()
+        return max(highest, seq or 0)
+    sequence = (await db.execute(text("SELECT pg_get_serial_sequence('spool', 'id')"))).scalar()
+    if sequence:
+        state = (await db.execute(text(f"SELECT last_value, is_called FROM {sequence}"))).first()  # noqa: S608  # nosec B608 -- name from pg_get_serial_sequence
+        if state is not None:
+            last_value, is_called = state
+            highest = max(highest, last_value if is_called else last_value - 1)
+    return highest
+
+
+async def _decide_ids(db: AsyncSession, rows: list[ImportRowResult], warnings: list[str]) -> None:
+    """Keep each valid row's id when this install never handed it out (#3520).
+
+    On a fresh install every id qualifies. An id already given out here, or
+    one the file names twice, would merge two spools' records, so that row
+    gets a new id instead: one warning for all of them, not one per row.
+    """
+    highest = await highest_spool_id_given_out(db)
+    taken: set[int] = set()
+    renumbered = 0
+    for row in rows:
+        if row.keeps_id is None:
+            continue
+        if row.status != "valid" or row.keeps_id <= highest or row.keeps_id in taken:
+            if row.status == "valid":
+                renumbered += 1
+            row.keeps_id = None
+            continue
+        taken.add(row.keeps_id)
+    if renumbered:
+        warnings.append(
+            f"{renumbered} spool(s) get a new id: their id is already used on this install or appears twice in the file"
+        )
+
+
 def serialize(spools: list[Spool]) -> bytes:
     """Render spools to CSV bytes using the fixed schema (export side).
 

+ 125 - 1
backend/tests/integration/test_inventory_csv.py

@@ -35,7 +35,7 @@ class TestInventoryCsvExport:
         body = response.text
         lines = body.strip().splitlines()
         # Header row uses the fixed schema.
-        assert lines[0].split(",")[0] == "material"
+        assert lines[0].split(",")[:2] == ["id", "material"]
         assert "rgba" in lines[0]
         # Data row present, rgba written without leading '#'.
         assert "Polymaker" in body
@@ -535,3 +535,127 @@ class TestInventoryCsvDuplicateWarning:
         assert real.json()["created"] == 2
         all_spools = (await db_session.execute(select(Spool))).scalars().all()
         assert len(all_spools) == 3  # 1 pre-existing + 2 imported
+
+
+@pytest.mark.asyncio
+@pytest.mark.integration
+class TestInventoryCsvKeepsIds:
+    """A moved inventory keeps its spool ids: printed labels' QR codes carry them (#3520)."""
+
+    async def _base(self, db_session: AsyncSession) -> int:
+        from backend.app.services.spool_csv import highest_spool_id_given_out
+
+        return await highest_spool_id_given_out(db_session)
+
+    async def _ids_by_color(self, db_session: AsyncSession) -> dict[str, int]:
+        db_session.expire_all()
+        rows = (await db_session.execute(select(Spool))).scalars().all()
+        return {s.color_name: s.id for s in rows}
+
+    async def test_export_writes_the_id(self, async_client: AsyncClient, db_session: AsyncSession):
+        spool = Spool(material="PLA", brand="B", color_name="Red", rgba="ff0000ff")
+        db_session.add(spool)
+        await db_session.commit()
+
+        response = await async_client.get("/api/v1/inventory/spools/export")
+
+        header, row = response.text.strip().splitlines()[:2]
+        assert header.split(",")[0] == "id"
+        assert row.split(",")[0] == str(spool.id)
+
+    async def test_import_keeps_ids_and_their_gaps(self, async_client: AsyncClient, db_session: AsyncSession):
+        base = await self._base(db_session)
+        # Out of order, with gaps, and a row without an id ahead of them: it
+        # must not take an id a later row still needs.
+        csv_text = f"id,material,color_name\n,PLA,New\n{base + 5},PLA,Five\n{base + 1},PETG,One\n{base + 3},PLA,Three\n"
+
+        response = await async_client.post("/api/v1/inventory/spools/import", files=_csv_upload(csv_text))
+
+        assert response.status_code == 200, response.text
+        assert response.json()["created"] == 4
+        ids = await self._ids_by_color(db_session)
+        assert ids["Five"] == base + 5
+        assert ids["One"] == base + 1
+        assert ids["Three"] == base + 3
+        assert ids["New"] == base + 6
+
+    async def test_the_next_spool_continues_after_the_kept_ids(
+        self, async_client: AsyncClient, db_session: AsyncSession
+    ):
+        """Only kept ids in the file: the next spool still gets a fresh id (PostgreSQL's sequence)."""
+        base = await self._base(db_session)
+        csv_text = f"id,material,color_name\n{base + 4},PLA,Four\n"
+        await async_client.post("/api/v1/inventory/spools/import", files=_csv_upload(csv_text))
+
+        response = await async_client.post("/api/v1/inventory/spools", json={"material": "PLA", "color_name": "Later"})
+
+        assert response.status_code == 200, response.text
+        assert response.json()["id"] == base + 5
+
+    async def test_dry_run_shows_the_id_each_spool_keeps(self, async_client: AsyncClient, db_session: AsyncSession):
+        base = await self._base(db_session)
+        csv_text = f"id,material\n{base + 2},PLA\n,PETG\n"
+
+        response = await async_client.post("/api/v1/inventory/spools/import?dry_run=true", files=_csv_upload(csv_text))
+
+        rows = response.json()["rows"]
+        assert [r["keeps_id"] for r in rows] == [base + 2, None]
+        assert response.json()["warnings"] == []
+
+    async def test_an_id_in_use_gets_a_new_one(self, async_client: AsyncClient, db_session: AsyncSession):
+        existing = Spool(material="PLA", color_name="Existing")
+        db_session.add(existing)
+        await db_session.commit()
+        csv_text = f"id,material,color_name\n{existing.id},PETG,Imported\n"
+
+        preview = await async_client.post("/api/v1/inventory/spools/import?dry_run=true", files=_csv_upload(csv_text))
+        await async_client.post("/api/v1/inventory/spools/import", files=_csv_upload(csv_text))
+
+        assert preview.json()["rows"][0]["keeps_id"] is None
+        assert any("1 spool(s) get a new id" in w for w in preview.json()["warnings"])
+        ids = await self._ids_by_color(db_session)
+        assert ids["Existing"] == existing.id
+        assert ids["Imported"] > existing.id
+
+    async def test_a_deleted_spools_id_is_not_reused(self, async_client: AsyncClient, db_session: AsyncSession):
+        """Old print logs and usage records may still name it."""
+        gone = Spool(material="PLA", color_name="Gone")
+        db_session.add(gone)
+        await db_session.commit()
+        gone_id = gone.id
+        await db_session.delete(gone)
+        await db_session.commit()
+        csv_text = f"id,material,color_name\n{gone_id},PLA,Imported\n"
+
+        await async_client.post("/api/v1/inventory/spools/import", files=_csv_upload(csv_text))
+
+        assert (await self._ids_by_color(db_session))["Imported"] > gone_id
+
+    async def test_an_id_named_twice_is_kept_once(self, async_client: AsyncClient, db_session: AsyncSession):
+        base = await self._base(db_session)
+        csv_text = f"id,material,color_name\n{base + 2},PLA,First\n{base + 2},PLA,Second\n"
+
+        response = await async_client.post("/api/v1/inventory/spools/import", files=_csv_upload(csv_text))
+
+        assert response.json()["created"] == 2
+        ids = await self._ids_by_color(db_session)
+        assert ids["First"] == base + 2
+        assert ids["Second"] == base + 3
+
+    @pytest.mark.parametrize("bad", ["abc", "0", "-3", "1.5"])
+    async def test_a_bad_id_is_a_row_error(self, async_client: AsyncClient, bad: str):
+        csv_text = f"id,material\n{bad},PLA\n"
+
+        response = await async_client.post("/api/v1/inventory/spools/import?dry_run=true", files=_csv_upload(csv_text))
+
+        row = response.json()["rows"][0]
+        assert row["status"] == "error"
+        assert "id must be a positive whole number" in row["reason"]
+
+    async def test_a_csv_without_an_id_column_still_imports(self, async_client: AsyncClient, db_session: AsyncSession):
+        response = await async_client.post(
+            "/api/v1/inventory/spools/import", files=_csv_upload("material,color_name\nPLA,Plain\n")
+        )
+
+        assert response.json()["created"] == 1
+        assert response.json()["errors"] == 0

+ 19 - 0
frontend/src/__tests__/components/SpoolCsvImportModal.test.tsx

@@ -55,6 +55,25 @@ describe('SpoolCsvImportModal', () => {
     expect(screen.getByText('Import 2 valid rows')).toBeInTheDocument();
   });
 
+  it('shows the id each spool keeps, and a dash for a new one (#3520)', async () => {
+    vi.mocked(api.importSpoolsCsvPreview).mockResolvedValue({
+      ...preview,
+      rows: [
+        { ...preview.rows[0], keeps_id: 417 },
+        { ...preview.rows[2], keeps_id: null },
+      ],
+    });
+
+    render(<SpoolCsvImportModal onClose={vi.fn()} onImported={vi.fn()} />);
+    selectFile();
+
+    expect(await screen.findByText('417')).toBeInTheDocument();
+    const rows = screen.getAllByRole('row');
+    // Header, then one row per CSV row; the id cell follows the row number.
+    expect(rows[1].querySelectorAll('td')[1].textContent).toBe('417');
+    expect(rows[2].querySelectorAll('td')[1].textContent).toBe('—');
+  });
+
   it('imports only when there are valid rows and reports the created count', async () => {
     vi.mocked(api.importSpoolsCsvPreview).mockResolvedValue(preview);
     vi.mocked(api.importSpoolsCsv).mockResolvedValue({ created: 2, skipped: 0, errors: 1, error_rows: [] });

+ 2 - 0
frontend/src/api/client.ts

@@ -4178,6 +4178,8 @@ export interface SpoolmanBulkCreateResult {
 export interface CsvImportRow {
   row_number: number;
   status: 'valid' | 'error' | 'skipped';
+  /** The id the spool is created with (#3520); null gets the next free one. */
+  keeps_id?: number | null;
   reason: string | null;
   material: string | null;
   brand: string | null;

+ 3 - 0
frontend/src/components/SpoolCsvImportModal.tsx

@@ -162,6 +162,7 @@ export function SpoolCsvImportModal({ onClose, onImported }: SpoolCsvImportModal
                       <thead className="bg-bambu-dark sticky top-0">
                         <tr className="text-left text-bambu-gray">
                           <th className="px-3 py-2 font-medium">{t('inventory.csv.colRow', 'Row')}</th>
+                          <th className="px-3 py-2 font-medium">#</th>
                           <th className="px-3 py-2 font-medium">{t('inventory.csv.colStatus', 'Status')}</th>
                           <th className="px-3 py-2 font-medium">{t('inventory.material', 'Material')}</th>
                           <th className="px-3 py-2 font-medium">{t('inventory.brand', 'Brand')}</th>
@@ -172,6 +173,8 @@ export function SpoolCsvImportModal({ onClose, onImported }: SpoolCsvImportModal
                         {preview.rows.map((row) => (
                           <tr key={row.row_number} className="border-t border-bambu-dark-tertiary">
                             <td className="px-3 py-2 text-bambu-gray">{row.row_number}</td>
+                            {/* The id the spool keeps (#3520); empty means it gets a new one. */}
+                            <td className="px-3 py-2 text-white">{row.keeps_id ?? '—'}</td>
                             <td className="px-3 py-2">
                               <div className="flex items-center gap-1.5">
                                 {statusIcon(row.status)}