Kaynağa Gözat

fix(db): stop holding pooled connections across FTP/camera/SMTP work (#2572)

The remaining routes of the idle-in-transaction class: the file-manager,
storage, camera-snapshot and timelapse routes each took their printer row
via Depends(get_db) and then talked FTP/camera on the same held session, so
a farm dashboard polling cover/snapshot tiles (offline printers included)
crept the pool to exhaustion over ~23h. They now read in a short session and
release before the I/O; timelapse re-opens a fresh session only for the write.

Also caps the four bare-executor FTP helpers with asyncio.wait_for so a
saturated 48-worker pool can't pin a caller (and its DB connection)
indefinitely, and runs the synchronous smtplib send off the event loop with
an explicit timeout so a wedged relay can't freeze the loop.
maziggy 1 ay önce
ebeveyn
işleme
cc75a24371

+ 3 - 0
CHANGELOG.md

@@ -14,6 +14,9 @@ All notable changes to Bambuddy will be documented in this file.
 - **Queue edit showed the sliced-for model as the scheduler target, and a cross-model queue row could dispatch G-code to an incompatible printer (#2578, reporter @Jostxxl)** — Two bugs with one root. The "Any \<model\>" assignment button labeled itself from the file's slice metadata while the scheduler actually used the row's `target_model`, so an X1C-sliced item targeting H2D read "Any X1C" above "Scheduler will assign to first available idle H2D printer". Worse, the mismatch could be *created* silently: the sliced-for model loads asynchronously, and clicking "Any Model" before it arrived pre-selected the first model alphabetically — on a mixed X1C/P1S/H2D farm that's H2D — after which the model dropdown hid itself, leaving no way to see or fix the wrong target. Nothing downstream checked compatibility, so the scheduler would happily hand X1C G-code to an H2D. Now: the target model is never silently defaulted (the dropdown stays visible in model mode, pre-selected to the sliced-for model when available, and back-fills once the metadata loads); the button reflects the actual target; a warning shows when the target differs from the sliced-for model. Compatibility is enforced end-to-end with an explicit G-code interchange family table (X1/X1C/X1E/P1P/P1S interchange; everything else exact-match — files without slice metadata are never blocked): incompatible models are disabled in the dropdown, queue create/update reject a mismatch with a clear 400 (so API-created rows can't sneak in), and the scheduler holds back pre-existing mismatched rows with an actionable waiting reason instead of dispatching them — fix the target via edit and the job flows again.
 - **Manual jog could drive an axis past its travel limit into a collision (#2579, reporter @R3play210)** — Jog the bed up from Bambuddy and, instead of stopping at the travel limit, it keeps going until the nozzle hits the plate; X/Y overrun too, on every model. Instrumenting the exact bytes sent to an H2D showed Bambuddy issuing a clean, correct move at the limit — `G91` / `G1 Z-1.00 F600` / `G90`, no endstop manipulation — that the printer executed straight past the stop, while the machine's **own touchscreen refuses the identical motion**. **This is a Bambu firmware bug: the firmware does not enforce its soft endstops on G-code received over MQTT** (the path every remote tool, Bambuddy included, must use), and it reports no axis position, so Bambuddy cannot know where the bed is to stop it either. It is not fixable from our side. Two things change here: (1) the jog no longer wraps moves in `M211 S0`/`S1` — the old code disabled the firmware's soft endstops *globally* around every jog, which also broke the **touchscreen's** limits until the printer was power-cycled; it now sends a bare move and never touches `M211`, so the touchscreen stays protected. (2) The jog panel now shows a prominent warning that travel limits are **not** enforced during manual moves because of this firmware bug, so nobody trusts the control to stop at the limit. Client-side travel-limit enforcement (dead-reckoning from a home) is tracked separately as the only real mitigation. If your printer currently overruns even from its touchscreen, power-cycle it once to restore the endstops an older Bambuddy build disabled.
 - **External spool kept its old inventory filament after the type was changed on the printer (#2575, reporter @ajbastien)** — Assigning a new filament to the external spool (e.g. generic ABS in place of generic TPU) left the previous inventory spool assigned, so an ABS spool stayed mapped to TPU. The reconciliation that unlinks a stale external-spool assignment lives in `on_ams_change`, but that callback only fired on changes to the regular AMS units — its change-hash never included the external spool (`vt_tray`/`vir_slot`), and the external-spool data is stored after the AMS handler runs. External-spool identity changes (type, colour, tag, or a reset to empty) now re-trigger the callback so the stale assignment is unlinked; the fill-percentage (`remain`) is deliberately excluded from the fingerprint so a running print doesn't fire it on every push. Follow-up: the auto-unlink now also broadcasts `spool_assignment_changed` for each cleared slot — previously only the manual assign/unassign endpoints did, so an open browser kept rendering the now-unlinked spool on the slot until an unrelated refetch, which read as "the fix didn't work" even though the server state was already correct (reporter confirmed a browser refresh showed the right state all along).
+- **The file-manager, storage, camera-snapshot and timelapse routes still held a DB connection across their FTP/camera work (#2572, reporter @Jostxxl)** — After the earlier #2572 fixes the farm still bled connections over a long run — the pool crept from its normal ~14 to the full 300 across ~23 hours (with only ~20 of 93 printers powered on) and then threw `QueuePool limit … connection timed out`. These were the remaining routes of the same class: each took its printer row via `Depends(get_db)`, whose session stays open for the whole request, and then talked FTP to the printer — a listing, a multi-MB download, a delete, a storage probe — with a browser polling the cover/snapshot tiles for every card, offline ones included, and 73 unreachable printers each burning a full FTP timeout. The printer-files endpoints (`/files`, `/files/download`, `/files/gcode`, `/files/plates`, `/files/plate-thumbnail`, `/files/download-zip`, `DELETE /files`, `/storage`), the camera **snapshot** endpoint (sibling of the already-fixed stream), and the timelapse **scan** and **select** endpoints now read what they need in a short session, release the connection *before* the FTP/camera work (`expire_on_commit=False` keeps the loaded `printer.*` columns readable), and — for timelapse, which also writes — re-open a fresh short session only to attach the downloaded file. Behaviour is unchanged; the timelapse-scan boundary is pinned by a regression test that mocks the FTP listing/download and asserts both the detached-row reads and that the attach persists through the fresh session. Completes the route-by-route half of the #2572 effort (camera stream, cover, on_print_start, timelapse scan, finish photo, notification snapshots).
+- **Four async FTP helpers had no overall timeout, so a saturated FTP thread-pool could pin a caller — and any DB connection it held — indefinitely (#2572, reporter @Jostxxl)** — FTP runs in a fixed 48-worker thread pool. `download_file_try_paths_async`, `download_file_bytes_async`, `get_storage_info_async` and `delete_file_async` wrapped their worker in a bare `run_in_executor` with no `asyncio.wait_for` (unlike `list_files_async`/`download_file_async`, which already had one). The per-socket timeout only bounds a worker once it *starts*; it does nothing for the time a call spends **queued** waiting for a free worker. On a farm where offline printers keep every worker parked on dead connects, that queue wait is unbounded — so an awaiting coroutine, and any pooled DB connection it was still holding, could wait forever. All four now cap the whole operation with `asyncio.wait_for` (returning the same failure sentinel on expiry, the orphaned worker's result discarded), so a backed-up FTP pool can no longer pin a caller — defence-in-depth beneath the route fixes above.
+- **A wedged SMTP server could freeze the entire event loop during an email notification (#2572, reporter @Jostxxl)** — `_send_email` ran `smtplib` **synchronously on the event loop** and constructed the connection with **no timeout** (smtplib then falls back to the global socket timeout, which the app never sets). A relay that accepts the TCP connection but stalls on the greeting/login/DATA left the send blocked forever — and because it ran inline, it stalled every other coroutine with it. The send now runs off the loop (`asyncio.to_thread`) with an explicit 30s connect timeout, and `quit()` moved into a `finally` so a mid-send error can't leak the socket. Latent bug surfaced while auditing #2572; it presents as a stall/latency spike rather than the pool leak, but the same "blocking I/O on the loop" family.
 - **The API didn't start serving for ~100 seconds on a large farm while it connected to printers one at a time (#2572, reporter @Jostxxl)** — On the reporter's 93-printer farm port 8000 didn't respond until roughly 100 seconds after the service started. The cause was in the FastAPI lifespan: `init_printer_connections` looped over every active printer and `await`ed each connection *serially*, and each `connect_printer` ends in a fixed one-second settle wait. The MQTT connect itself is non-blocking — `BambuMQTTClient.connect()` only calls `connect_async()` + `loop_start()`, so the handshake runs on a background thread — which means that one-second wait, times the fleet size, was pure serial dead air that the lifespan blocked on *before* the ASGI server began accepting requests. The connections are now started concurrently with `asyncio.gather`, so the whole step takes about a second regardless of how many printers you run, and the dashboard is reachable almost immediately. Each connection's result is also isolated (`return_exceptions=True`): a single unreachable printer no longer aborts the rest — or, as the old un-guarded serial `await` allowed, the entire startup. The MQTT clients still connect in the background exactly as before; only the startup wait is parallelized.
 - **The print-start handler held a DB connection open across plate detection and the 3MF download (#2572, reporter @Jostxxl)** — After farm-testing the first round of #2572 fixes the reporter still saw `idle in transaction` sessions lasting minutes, and traced one to `on_print_start`: its last statement was `SELECT print_archives…`, immediately followed in the log by the printer's own `on_print_start` → `Trying filenames` → FTP work. The handler opened a single database session at the top and held it to the very end of the function — across two slow I/O blocks that need no database: the optional plate-detection camera capture (a 2.5s chamber-light settle plus an FTP/RTSP grab) and, on the new-archive path, the 3MF FTP download itself (up to five remote paths per candidate filename, each with retry/backoff — the code's own comments cite worst cases of tens of minutes under FTP contention). So one pooled connection sat idle-in-transaction for the whole of both, once per starting print, and print starts cluster on a farm. The connection is now released at both boundaries: reaching either point, only read `SELECT`s have run on that path (every write branch returns earlier), so a commit persists nothing and simply ends the read transaction, returning the connection to the pool for the duration of the I/O; the next query re-acquires a fresh one, and `expire_on_commit=False` keeps the already-loaded `printer.*` columns readable with no lazy load. Behaviour is unchanged. Continues the #2572 effort (camera stream, timelapse scan, finish photo, notification snapshots) to stop holding sessions across slow I/O.
 - **The printer-cover endpoint held a DB connection open across the FTP thumbnail download (#2572, reporter @Jostxxl)** — The reporter's second correlation: a transaction whose last statement was `SELECT printers…`, matched in the log to the cover route (`Cover: resolved plate …` / `Trying to download cover … (trying 4 paths)`), still open more than three and a half minutes later. `GET /printers/{id}/cover` took its printer row via `Depends(get_db)`, and `get_db` is a `yield` dependency — its session stays open for the whole request, including the cover's 3MF download (up to eight remote paths × retries with backoff, minutes under the same single-FTP-socket contention that produces the 425s). The session was used for exactly one `SELECT`; everything after reads already-loaded `printer.*` scalars, `printer_manager`, and FTP/zip — no database. The endpoint now fetches the printer in a short-lived session and releases the connection *before* the download (`expire_on_commit=False` keeps the columns readable), mirroring the camera-stream fix. Pinned by a regression test that fails if a `get_db`-held session is ever re-added to the route.

+ 39 - 28
backend/app/api/routes/archives.py

@@ -2247,10 +2247,10 @@ async def delete_timelapse(
 @router.post("/{archive_id}/timelapse/scan")
 async def scan_timelapse(
     archive_id: int,
-    db: AsyncSession = Depends(get_db),
     _: User | None = RequirePermissionIfAuthEnabled(Permission.ARCHIVES_UPDATE_ALL),
 ):
     """Scan printer for timelapse matching this archive and attach it."""
+    from backend.app.core.database import async_session
     from backend.app.models.printer import Printer
     from backend.app.services.bambu_ftp import (
         download_file_bytes_async,
@@ -2259,22 +2259,27 @@ async def scan_timelapse(
         with_ftp_retry,
     )
 
-    service = ArchiveService(db)
-    archive = await service.get_archive(archive_id)
-    if not archive:
-        raise HTTPException(404, "Archive not found")
+    # Read the archive + printer in a short session and release the pooled DB
+    # connection BEFORE the FTP scan/download below — a timelapse pull walks
+    # several directories and fetches a 100MB+ video, so holding Depends(get_db)
+    # across it pinned one connection idle-in-transaction for minutes (#2572).
+    # Scalar columns stay readable on the detached rows (expire_on_commit=False);
+    # the attach at the end runs in its own fresh short session.
+    async with async_session() as db:
+        archive = await ArchiveService(db).get_archive(archive_id)
+        if not archive:
+            raise HTTPException(404, "Archive not found")
 
-    if archive.timelapse_path:
-        return {"status": "exists", "message": "Timelapse already attached"}
+        if archive.timelapse_path:
+            return {"status": "exists", "message": "Timelapse already attached"}
 
-    if not archive.printer_id:
-        raise HTTPException(400, "Archive has no associated printer")
+        if not archive.printer_id:
+            raise HTTPException(400, "Archive has no associated printer")
 
-    # Get printer
-    result = await db.execute(select(Printer).where(Printer.id == archive.printer_id))
-    printer = result.scalar_one_or_none()
-    if not printer:
-        raise HTTPException(404, "Printer not found")
+        result = await db.execute(select(Printer).where(Printer.id == archive.printer_id))
+        printer = result.scalar_one_or_none()
+        if not printer:
+            raise HTTPException(404, "Printer not found")
 
     # Get base name from archive filename (without .3mf extension)
     base_name = Path(archive.filename).stem
@@ -2413,8 +2418,9 @@ async def scan_timelapse(
     if not timelapse_data:
         raise HTTPException(500, "Failed to download timelapse")
 
-    # Attach timelapse to archive
-    success = await service.attach_timelapse(archive_id, timelapse_data, matching_file["name"])
+    # Attach in a fresh short session (the read session was released before FTP).
+    async with async_session() as db:
+        success = await ArchiveService(db).attach_timelapse(archive_id, timelapse_data, matching_file["name"])
 
     if not success:
         raise HTTPException(500, "Failed to attach timelapse")
@@ -2430,10 +2436,10 @@ async def scan_timelapse(
 async def select_timelapse(
     archive_id: int,
     filename: str = Query(..., description="Timelapse filename to attach"),
-    db: AsyncSession = Depends(get_db),
     _: User | None = RequirePermissionIfAuthEnabled(Permission.ARCHIVES_UPDATE_ALL),
 ):
     """Manually select a timelapse from the printer to attach."""
+    from backend.app.core.database import async_session
     from backend.app.models.printer import Printer
     from backend.app.services.bambu_ftp import (
         download_file_bytes_async,
@@ -2442,18 +2448,21 @@ async def select_timelapse(
         with_ftp_retry,
     )
 
-    service = ArchiveService(db)
-    archive = await service.get_archive(archive_id)
-    if not archive:
-        raise HTTPException(404, "Archive not found")
+    # Read the archive + printer in a short session and release the pooled DB
+    # connection BEFORE the FTP scan/download below (#2572); scalars stay
+    # readable after close (expire_on_commit=False), the attach reopens one.
+    async with async_session() as db:
+        archive = await ArchiveService(db).get_archive(archive_id)
+        if not archive:
+            raise HTTPException(404, "Archive not found")
 
-    if not archive.printer_id:
-        raise HTTPException(400, "Archive has no associated printer")
+        if not archive.printer_id:
+            raise HTTPException(400, "Archive has no associated printer")
 
-    result = await db.execute(select(Printer).where(Printer.id == archive.printer_id))
-    printer = result.scalar_one_or_none()
-    if not printer:
-        raise HTTPException(404, "Printer not found")
+        result = await db.execute(select(Printer).where(Printer.id == archive.printer_id))
+        printer = result.scalar_one_or_none()
+        if not printer:
+            raise HTTPException(404, "Printer not found")
 
     # Find the file on the printer
     files = []
@@ -2502,7 +2511,9 @@ async def select_timelapse(
     if not timelapse_data:
         raise HTTPException(500, "Failed to download timelapse")
 
-    success = await service.attach_timelapse(archive_id, timelapse_data, filename)
+    # Attach in a fresh short session (the read session was released before FTP).
+    async with async_session() as db:
+        success = await ArchiveService(db).attach_timelapse(archive_id, timelapse_data, filename)
     if not success:
         raise HTTPException(500, "Failed to attach timelapse")
 

+ 9 - 2
backend/app/api/routes/camera.py

@@ -889,7 +889,6 @@ async def stop_camera_stream(
 @router.get("/{printer_id}/camera/snapshot")
 async def camera_snapshot(
     printer_id: int,
-    db: AsyncSession = Depends(get_db),
     _: None = RequireCameraStreamTokenIfAuthEnabled,
 ):
     """Capture a single frame from the printer camera.
@@ -901,7 +900,15 @@ async def camera_snapshot(
     import tempfile
     from pathlib import Path
 
-    printer = await get_printer_or_404(printer_id, db)
+    # Fetch the printer in a short-lived session and release the pooled DB
+    # connection BEFORE the camera capture below (up to 15s, longer under a
+    # saturated FTP/camera pool). Holding a Depends(get_db) session across the
+    # grab pinned one connection per snapshot — and the cam wall polls this
+    # per tile every 8s — so overlapping captures could pile up connections on
+    # a large farm (issue #2572, sibling of the camera_stream fix). Everything
+    # below reads only already-loaded scalar columns (expire_on_commit=False).
+    async with database.async_session() as db:
+        printer = await get_printer_or_404(printer_id, db)
 
     # Check for external camera first
     if printer.external_camera_enabled and printer.external_camera_url:

+ 31 - 42
backend/app/api/routes/printers.py

@@ -1212,18 +1212,37 @@ async def get_printer_cover(
 # ============================================
 
 
+async def _load_printer_or_404(printer_id: int) -> Printer:
+    """Load a printer in a short-lived session, releasing the pooled DB
+    connection before the caller starts any FTP/network I/O (#2572).
+
+    The file-manager and storage routes talk FTP to the printer, which can
+    block for the full socket timeout — longer when a saturated FTP pool backs
+    up. Holding the request's Depends(get_db) session across that FTP pinned one
+    pooled connection idle-in-transaction per in-flight request, a top cause of
+    pool exhaustion on large farms. The returned row's scalar columns stay
+    readable after the session closes (expire_on_commit=False). Raises 404 when
+    the printer doesn't exist.
+
+    Reference async_session via the module so the maker is resolved at call time
+    — keeps it in sync with reinitialize_database() and lets tests patch it.
+    """
+    async with database.async_session() as db:
+        result = await db.execute(select(Printer).where(Printer.id == printer_id))
+        printer = result.scalar_one_or_none()
+    if not printer:
+        raise HTTPException(404, "Printer not found")
+    return printer
+
+
 @router.get("/{printer_id}/files")
 async def list_printer_files(
     printer_id: int,
     path: str = "/",
     _=RequirePermissionIfAuthEnabled(Permission.PRINTERS_FILES),
-    db: AsyncSession = Depends(get_db),
 ):
     """List files on the printer at the specified path."""
-    result = await db.execute(select(Printer).where(Printer.id == printer_id))
-    printer = result.scalar_one_or_none()
-    if not printer:
-        raise HTTPException(404, "Printer not found")
+    printer = await _load_printer_or_404(printer_id)
 
     files = await list_files_async(printer.ip_address, printer.access_code, path, printer_model=printer.model)
 
@@ -1242,13 +1261,9 @@ async def download_printer_file(
     printer_id: int,
     path: str,
     _=RequirePermissionIfAuthEnabled(Permission.PRINTERS_FILES),
-    db: AsyncSession = Depends(get_db),
 ):
     """Download a file from the printer."""
-    result = await db.execute(select(Printer).where(Printer.id == printer_id))
-    printer = result.scalar_one_or_none()
-    if not printer:
-        raise HTTPException(404, "Printer not found")
+    printer = await _load_printer_or_404(printer_id)
 
     data = await download_file_bytes_async(printer.ip_address, printer.access_code, path, printer_model=printer.model)
     if data is None:
@@ -1283,16 +1298,11 @@ async def get_printer_file_gcode(
     printer_id: int,
     path: str,
     _=RequirePermissionIfAuthEnabled(Permission.PRINTERS_FILES),
-    db: AsyncSession = Depends(get_db),
 ):
     """Get gcode for a file stored on a printer (for preview)."""
     import io
 
-    # Validate printer
-    result = await db.execute(select(Printer).where(Printer.id == printer_id))
-    printer = result.scalar_one_or_none()
-    if not printer:
-        raise HTTPException(404, "Printer not found")
+    printer = await _load_printer_or_404(printer_id)
 
     data = await download_file_bytes_async(printer.ip_address, printer.access_code, path, printer_model=printer.model)
     if data is None:
@@ -1322,7 +1332,6 @@ async def get_printer_file_plates(
     printer_id: int,
     path: str = Query(..., description="Full path to the 3MF file on the printer"),
     _=RequirePermissionIfAuthEnabled(Permission.PRINTERS_FILES),
-    db: AsyncSession = Depends(get_db),
 ):
     """Get available plates from a multi-plate 3MF file stored on a printer."""
     import io
@@ -1330,11 +1339,7 @@ async def get_printer_file_plates(
 
     import defusedxml.ElementTree as ET
 
-    # Validate printer
-    result = await db.execute(select(Printer).where(Printer.id == printer_id))
-    printer = result.scalar_one_or_none()
-    if not printer:
-        raise HTTPException(404, "Printer not found")
+    printer = await _load_printer_or_404(printer_id)
 
     filename = path.split("/")[-1]
     if not filename.lower().endswith(".3mf"):
@@ -1567,15 +1572,11 @@ async def get_printer_file_plate_thumbnail(
     plate_index: int,
     path: str = Query(..., description="Full path to the 3MF file on the printer"),
     _=RequirePermissionIfAuthEnabled(Permission.PRINTERS_FILES),
-    db: AsyncSession = Depends(get_db),
 ):
     """Get a plate thumbnail image from a printer-stored 3MF file."""
     import io
 
-    result = await db.execute(select(Printer).where(Printer.id == printer_id))
-    printer = result.scalar_one_or_none()
-    if not printer:
-        raise HTTPException(404, "Printer not found")
+    printer = await _load_printer_or_404(printer_id)
 
     data = await download_file_bytes_async(printer.ip_address, printer.access_code, path, printer_model=printer.model)
     if data is None:
@@ -1598,7 +1599,6 @@ async def download_printer_files_as_zip(
     printer_id: int,
     request: dict,
     _=RequirePermissionIfAuthEnabled(Permission.PRINTERS_FILES),
-    db: AsyncSession = Depends(get_db),
 ):
     """Download multiple files from the printer as a ZIP archive."""
     import io
@@ -1607,10 +1607,7 @@ async def download_printer_files_as_zip(
     if not paths:
         raise HTTPException(400, "No files specified")
 
-    result = await db.execute(select(Printer).where(Printer.id == printer_id))
-    printer = result.scalar_one_or_none()
-    if not printer:
-        raise HTTPException(404, "Printer not found")
+    printer = await _load_printer_or_404(printer_id)
 
     # Create ZIP in memory
     zip_buffer = io.BytesIO()
@@ -1645,13 +1642,9 @@ async def delete_printer_file(
     printer_id: int,
     path: str,
     _=RequirePermissionIfAuthEnabled(Permission.PRINTERS_FILES),
-    db: AsyncSession = Depends(get_db),
 ):
     """Delete a file from the printer."""
-    result = await db.execute(select(Printer).where(Printer.id == printer_id))
-    printer = result.scalar_one_or_none()
-    if not printer:
-        raise HTTPException(404, "Printer not found")
+    printer = await _load_printer_or_404(printer_id)
 
     from backend.app.services.bambu_ftp import DeleteResult
 
@@ -1668,13 +1661,9 @@ async def delete_printer_file(
 async def get_printer_storage(
     printer_id: int,
     _=RequirePermissionIfAuthEnabled(Permission.PRINTERS_READ),
-    db: AsyncSession = Depends(get_db),
 ):
     """Get storage information from the printer."""
-    result = await db.execute(select(Printer).where(Printer.id == printer_id))
-    printer = result.scalar_one_or_none()
-    if not printer:
-        raise HTTPException(404, "Printer not found")
+    printer = await _load_printer_or_404(printer_id)
 
     storage_info = await get_storage_info_async(printer.ip_address, printer.access_code, printer_model=printer.model)
 

+ 42 - 4
backend/app/services/bambu_ftp.py

@@ -1009,12 +1009,21 @@ async def download_file_try_paths_async(
     local_path: Path,
     socket_timeout: float | None = None,
     printer_model: str | None = None,
+    timeout: float = 90.0,
 ) -> bool:
     """Try downloading a file from multiple paths using a single connection.
 
     Args:
         socket_timeout: FTP socket timeout for slow connections (e.g., A1 printers)
         printer_model: Printer model for A1-specific workarounds
+        timeout: overall async cap. The per-socket timeout only bounds an
+            in-flight worker; it does NOT bound how long this coroutine waits
+            for a free slot in the fixed-size ``_ftp_executor``. On a large
+            farm where offline printers keep every worker busy on dead
+            connects, that queue wait is otherwise unbounded — and any caller
+            holding a DB connection while awaiting this would pin it until the
+            pool is exhausted (#2572). The cap converts that into a bounded
+            wait; the orphaned worker finishes and its result is discarded.
     """
     loop = asyncio.get_event_loop()
 
@@ -1037,7 +1046,11 @@ async def download_file_try_paths_async(
         finally:
             client.disconnect()
 
-    return await loop.run_in_executor(_ftp_executor, _download)
+    try:
+        return await asyncio.wait_for(loop.run_in_executor(_ftp_executor, _download), timeout=timeout)
+    except TimeoutError:
+        logger.warning("FTP download_try_paths exceeded its %ss cap for %s (#2572)", timeout, ip_address)
+        return False
 
 
 def _upload_deadline(local_path: Path) -> float:
@@ -1249,6 +1262,7 @@ async def delete_file_async(
     remote_path: str,
     socket_timeout: float | None = None,
     printer_model: str | None = None,
+    timeout: float = 60.0,
 ) -> DeleteResult:
     """Async wrapper for deleting a file.
 
@@ -1259,6 +1273,8 @@ async def delete_file_async(
     Args:
         socket_timeout: FTP socket timeout for slow connections (e.g., A1 printers)
         printer_model: Printer model for A1-specific workarounds
+        timeout: overall async cap so a saturated ``_ftp_executor`` can't pin
+            the caller (and any DB connection it holds) indefinitely (#2572).
     """
     loop = asyncio.get_event_loop()
 
@@ -1271,7 +1287,11 @@ async def delete_file_async(
                 client.disconnect()
         return DeleteResult.FAILED
 
-    return await loop.run_in_executor(_ftp_executor, _delete)
+    try:
+        return await asyncio.wait_for(loop.run_in_executor(_ftp_executor, _delete), timeout=timeout)
+    except TimeoutError:
+        logger.warning("FTP delete_file exceeded its %ss cap for %s (#2572)", timeout, ip_address)
+        return DeleteResult.FAILED
 
 
 async def download_file_bytes_async(
@@ -1280,12 +1300,19 @@ async def download_file_bytes_async(
     remote_path: str,
     socket_timeout: float | None = None,
     printer_model: str | None = None,
+    timeout: float = 300.0,
 ) -> bytes | None:
     """Async wrapper for downloading file as bytes.
 
     Args:
         socket_timeout: FTP socket timeout for slow connections (e.g., A1 printers)
         printer_model: Printer model for A1-specific workarounds
+        timeout: overall async cap so a saturated ``_ftp_executor`` can't pin
+            the caller (and any DB connection it holds) indefinitely (#2572).
+            Generous by default because this pulls whole files (timelapse
+            video, gcode) which can legitimately take minutes over slow Wi-Fi —
+            the cap only guards against a permanently-starved pool, not a
+            slow-but-progressing transfer.
     """
     loop = asyncio.get_event_loop()
 
@@ -1298,7 +1325,11 @@ async def download_file_bytes_async(
                 client.disconnect()
         return None
 
-    return await loop.run_in_executor(_ftp_executor, _download)
+    try:
+        return await asyncio.wait_for(loop.run_in_executor(_ftp_executor, _download), timeout=timeout)
+    except TimeoutError:
+        logger.warning("FTP download_bytes exceeded its %ss cap for %s (#2572)", timeout, ip_address)
+        return None
 
 
 async def get_storage_info_async(
@@ -1306,12 +1337,15 @@ async def get_storage_info_async(
     access_code: str,
     socket_timeout: float | None = None,
     printer_model: str | None = None,
+    timeout: float = 60.0,
 ) -> dict | None:
     """Async wrapper for getting storage info.
 
     Args:
         socket_timeout: FTP socket timeout for slow connections (e.g., A1 printers)
         printer_model: Printer model for A1-specific workarounds
+        timeout: overall async cap so a saturated ``_ftp_executor`` can't pin
+            the caller (and any DB connection it holds) indefinitely (#2572).
     """
     loop = asyncio.get_event_loop()
 
@@ -1324,7 +1358,11 @@ async def get_storage_info_async(
                 client.disconnect()
         return None
 
-    return await loop.run_in_executor(_ftp_executor, _get_storage)
+    try:
+        return await asyncio.wait_for(loop.run_in_executor(_ftp_executor, _get_storage), timeout=timeout)
+    except TimeoutError:
+        logger.warning("FTP get_storage_info exceeded its %ss cap for %s (#2572)", timeout, ip_address)
+        return None
 
 
 async def get_ftp_retry_settings() -> tuple[bool, int, float, float]:

+ 37 - 16
backend/app/services/notification_service.py

@@ -512,22 +512,43 @@ class NotificationService:
                 msg["Subject"] = f"[Bambuddy] {subject}"
                 msg.attach(MIMEText(body, "plain"))
 
-            if security == "ssl":
-                # Direct SSL connection (typically port 465)
-                server = smtplib.SMTP_SSL(smtp_server, smtp_port)
-            elif security == "starttls":
-                # STARTTLS upgrade (typically port 587)
-                server = smtplib.SMTP(smtp_server, smtp_port)
-                server.starttls()
-            else:
-                # No encryption (typically port 25) - use with caution
-                server = smtplib.SMTP(smtp_server, smtp_port)
-
-            if auth_enabled:
-                server.login(username, password)
-
-            server.sendmail(from_email, to_email, msg.as_string())
-            server.quit()
+            # smtplib is synchronous and blocking: a wedged / greylisting /
+            # firewall-dropped relay leaves recv() stuck. Two problems, two
+            # fixes (#2572):
+            #   1. No timeout — smtplib defaults to the global socket timeout,
+            #      which this app never sets, so a stuck relay blocks forever.
+            #      Pass an explicit timeout to every connect.
+            #   2. Run on the event loop — a stuck (or merely slow) send freezes
+            #      every other coroutine, including a DB session a caller is
+            #      holding open across this notification. Offload to a worker
+            #      thread so the loop stays live and the connection is released
+            #      on schedule.
+            smtp_timeout = 30.0
+            msg_str = msg.as_string()
+
+            def _blocking_send() -> None:
+                if security == "ssl":
+                    # Direct SSL connection (typically port 465)
+                    server = smtplib.SMTP_SSL(smtp_server, smtp_port, timeout=smtp_timeout)
+                elif security == "starttls":
+                    # STARTTLS upgrade (typically port 587)
+                    server = smtplib.SMTP(smtp_server, smtp_port, timeout=smtp_timeout)
+                    server.starttls()
+                else:
+                    # No encryption (typically port 25) - use with caution
+                    server = smtplib.SMTP(smtp_server, smtp_port, timeout=smtp_timeout)
+                try:
+                    if auth_enabled:
+                        server.login(username, password)
+                    server.sendmail(from_email, to_email, msg_str)
+                finally:
+                    # quit() in finally so a send error doesn't leak the socket.
+                    try:
+                        server.quit()
+                    except Exception:  # noqa: BLE001 — closing a broken connection is best-effort
+                        pass
+
+            await asyncio.to_thread(_blocking_send)
 
             return True, "Email sent successfully"
         except smtplib.SMTPAuthenticationError:

+ 137 - 0
backend/tests/integration/test_timelapse_scan_session.py

@@ -0,0 +1,137 @@
+"""Regression tests for the #2572 timelapse-scan session-boundary refactor.
+
+``POST /archives/{id}/timelapse/scan`` used to hold its ``Depends(get_db)``
+session open across the FTP directory listing *and* the multi-MB video
+download. It now (1) reads the archive + printer in a short session and
+releases the pooled connection *before* the FTP work, then (2) re-opens a
+fresh short session only to attach the downloaded file.
+
+Two things that refactor could have broken, one test each:
+
+* The matching logic reads ``archive.filename/started_at/completed_at/
+  created_at`` and ``printer.ip_address/...`` AFTER the read session has
+  closed. If any were a lazy-loaded relationship (or an expired column) that
+  would raise ``DetachedInstanceError``. The not-found test drives every
+  match strategy, exercising all of those detached reads.
+
+* The attach write runs in a *fresh* ``async_session()``, which — unlike
+  ``get_db`` — does NOT auto-commit on block exit. If ``attach_timelapse``
+  didn't commit internally the write would be silently dropped. The attach
+  test asserts the row is actually persisted.
+
+FTP is fully mocked, so no printer is contacted.
+"""
+
+from __future__ import annotations
+
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+from httpx import AsyncClient
+
+
+@pytest.mark.asyncio
+@pytest.mark.integration
+async def test_scan_timelapse_no_match_reads_detached_archive_scalars(
+    async_client: AsyncClient, archive_factory, printer_factory, db_session
+):
+    """Two non-matching videos → 200 not_found, driving every match strategy.
+
+    Strategies 2-4 read archive.started_at/completed_at/created_at after the
+    read session closed; this fails with DetachedInstanceError if the refactor
+    left one of those as a lazy load.
+    """
+    printer = await printer_factory()
+    archive = await archive_factory(printer.id, filename="test_print.gcode.3mf")
+
+    # Two videos, neither matching by name, no mtime, and the archive has no
+    # started_at — so strategy 1 (name) misses, 2 (start time) and 3 (mtime)
+    # are skipped, and 4 (single-file fallback) is disqualified by len == 2.
+    listing = [
+        {"name": "clip_a.mp4", "path": "/timelapse/clip_a.mp4", "is_directory": False, "size": 10, "mtime": None},
+        {"name": "clip_b.mp4", "path": "/timelapse/clip_b.mp4", "is_directory": False, "size": 20, "mtime": None},
+    ]
+
+    with (
+        patch("backend.app.services.bambu_ftp.list_files_async", AsyncMock(return_value=listing)),
+        patch(
+            "backend.app.services.bambu_ftp.get_ftp_retry_settings",
+            AsyncMock(return_value=(False, 3, 2.0, 30.0)),
+        ),
+        patch(
+            "backend.app.services.bambu_ftp.download_file_bytes_async",
+            AsyncMock(return_value=b"should-not-be-called"),
+        ) as mock_download,
+    ):
+        response = await async_client.post(f"/api/v1/archives/{archive.id}/timelapse/scan")
+
+    assert response.status_code == 200, response.text
+    data = response.json()
+    assert data["status"] == "not_found"
+    assert {f["name"] for f in data["available_files"]} == {"clip_a.mp4", "clip_b.mp4"}
+    # No match → we never download.
+    mock_download.assert_not_called()
+
+
+@pytest.mark.asyncio
+@pytest.mark.integration
+async def test_scan_timelapse_attaches_and_persists_via_fresh_session(
+    async_client: AsyncClient, archive_factory, printer_factory, db_session, tmp_path, monkeypatch
+):
+    """A name-matched video is downloaded and the attach PERSISTS.
+
+    Guards the fresh-session write boundary: attach_timelapse runs in a new
+    async_session that does not auto-commit on exit, so this only passes if
+    the service commits internally.
+    """
+    printer = await printer_factory()
+    archive = await archive_factory(printer.id, filename="test_print.gcode.3mf")
+
+    # attach_timelapse writes into settings.base_dir / archive.file_path's
+    # parent, then stores a base_dir-relative timelapse_path. Point base_dir at
+    # tmp and stage the archive dir so the real write succeeds (mirrors
+    # test_attach_timelapse_safe_path).
+    monkeypatch.setattr(
+        "backend.app.services.archive.settings",
+        MagicMock(base_dir=tmp_path),
+    )
+    archive_dir = tmp_path / "archives" / "test"
+    archive_dir.mkdir(parents=True)
+
+    # base_name = Path("test_print.gcode.3mf").stem = "test_print.gcode", so this
+    # video matches by name (strategy 1). .mp4 → no background conversion task.
+    matched = {
+        "name": "test_print.gcode.mp4",
+        "path": "/timelapse/test_print.gcode.mp4",
+        "is_directory": False,
+        "size": 4096,
+        "mtime": None,
+    }
+    video_bytes = b"fake-timelapse-video-bytes"
+
+    with (
+        patch("backend.app.services.bambu_ftp.list_files_async", AsyncMock(return_value=[matched])),
+        patch(
+            "backend.app.services.bambu_ftp.get_ftp_retry_settings",
+            AsyncMock(return_value=(False, 3, 2.0, 30.0)),
+        ),
+        patch(
+            "backend.app.services.bambu_ftp.download_file_bytes_async",
+            AsyncMock(return_value=video_bytes),
+        ) as mock_download,
+    ):
+        response = await async_client.post(f"/api/v1/archives/{archive.id}/timelapse/scan")
+
+    assert response.status_code == 200, response.text
+    data = response.json()
+    assert data["status"] == "attached"
+    assert data["filename"] == "test_print.gcode.mp4"
+    mock_download.assert_awaited_once()
+
+    # The write happened in the route's fresh session; confirm it was committed
+    # by re-reading the row on the separate test session.
+    await db_session.refresh(archive)
+    assert archive.timelapse_path is not None
+    assert archive.timelapse_path.endswith("test_print.gcode.mp4")
+    # And the bytes actually landed on disk under the staged archive dir.
+    assert (archive_dir / "test_print.gcode.mp4").read_bytes() == video_bytes

+ 5 - 1
backend/tests/unit/services/test_notification_service.py

@@ -2379,9 +2379,13 @@ class TestEmailProvider:
     @staticmethod
     def _fake_smtp_class(captured: dict):
         class FakeSMTP:
-            def __init__(self, host, port):
+            # timeout matches the real smtplib.SMTP/SMTP_SSL signature — the
+            # service passes an explicit timeout so a wedged relay can't hang
+            # the send (#2572).
+            def __init__(self, host, port, timeout=None):
                 captured["host"] = host
                 captured["port"] = port
+                captured["timeout"] = timeout
 
             def starttls(self):
                 captured["starttls"] = True