Shrink large images, reporting wherever the caller already reports (BL-16646) - #8154
Shrink large images, reporting wherever the caller already reports (BL-16646)#8154StephenMcConnel wants to merge 17 commits into
Conversation
MigrateToMediaLevel1ShrinkLargeImages shows an "Updating Image Files" ProgressDialogBackground when a book still has images larger than 3840x2800. Several of the operations that bring a book up to date run on a background thread -- Do Updates of All Books, the BloomPub/ePUB publish preview, .bloomSource and spreadsheet import, the bulk-upload CLI -- and WinForms does not allow a Form to be created off the UI thread. So on an old book that still has an oversized image, those paths could hang, throw, or leave a stray dialog. Marshal the dialog onto the UI thread via Shell.GetShellOrOtherOpenForm() and Invoke, the same pattern already used in AudioRecording, NonFatalProblem, and OneDriveUtils. When no form is open at all -- the headless CLI, which has no message loop -- do the work directly against a NullProgress instead of trying to show a modal dialog that nobody could see or dismiss. The dialog body moves unchanged into a new UpdateImagesWithProgressDialog helper. mediaMaintenanceLevel is still bumped in exactly the same place for every branch, and the Program.RunningUnitTests special case is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…(BL-16646) The new helper is the one piece of this code that must not be called off the UI thread, which is the whole point of the branch above it. Say so where someone editing the helper will see it, per our convention of commenting private methods. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
[Claude Opus 5 (1M context) during preflight] Consulted Devin on 2026-08-05 up to commit It found no bugs. It raised one thing worth investigating — whether the blocking Two further items were informational and not mirrored as threads, but both bear on the design question raised in the PR description, so they are worth a reviewer's eye: that CI ( |
… (BL-16646)
MigrateToMediaLevel1ShrinkLargeImages used to put up its own "Updating Image
Files" ProgressDialogBackground. Several of the operations that bring a book up
to date run on a background thread, and WinForms does not allow a Form to be
created there, so on an old book that still had an image larger than 3840x2800
those paths could hang, throw, or leave a stray dialog.
Rather than teach the dialog which thread it is on, remove it. The migration now
takes the IProgress its caller already has and hands that to
ImageUtils.FixSizeAndTransparencyOfImagesInFolder, which has always accepted one.
With no Form there is no thread rule left to break, and this moves with the
conversion to React instead of adding to what has to be unwound later.
Each caller now gets the progress it already had: Do Updates of All Books and the
.bloomSource import and publish-preview paths report into the progress dialog
they had already opened, while headless callers such as the bulk-upload CLI pass
NullProgress and stay silent rather than trying to show a modal that nobody could
see or dismiss. The user still gets per-image feedback either way, because the
resize itself writes the already-localized "Preparing image: {0}" status.
The Program.RunningUnitTests special case goes away with the dialog. It only ever
existed to dodge the modal, and both of its branches now collapse into the same
single call.
Also widen WebProgressAdapter to accept IWebSocketProgress rather than the
concrete WebSocketProgress, so MakeDerivativeFromBloomSourceFile can forward the
progress it is handed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…-16646) Removing the dialog left two comments describing behavior that no longer happens: the block comment in MigrateToMediaLevel1ShrinkLargeImages still said a progress dialog pops up when images must be shrunk, and the method's doc comment said nothing about the new IProgress parameter at all. Say what actually happens instead, and document the parameter -- including that headless callers pass NullProgress and so report nothing, and that it must not be null because ImageUtils.FixSizeAndTransparencyOfImagesInFolder dereferences it. (Book.EnsureUpToDate already substitutes a NullProgress for a null argument, which is what keeps every current caller safe.) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
[Claude Opus 5 (1M context) during preflight] Consulted Devin on 2026-08-05 up to commit This was a re-review after the branch was reworked to delete the progress dialog rather than marshal it onto the UI thread. No bugs. Three things needed investigating and four were informational. The one that matters is the loss of feedback on the Of the rest, the stale PR description was a fair catch that had already been fixed earlier in this run, and is resolved. Devin also re-raised the blocking- The four informational items were not mirrored: a missing null guard on the new CI ( |
…L-16646) Design 2 removed the "Updating Image Files" dialog entirely and reported through the caller's IProgress. That left the callers holding only a NullProgress with no feedback at all, most visibly book selection. This reinstates the dialog for the callers that have nothing better, while letting the ones that own a real browser progress (WebProgressAdapter) keep reporting into it, and keeps the direct call for unit tests. Also guard against a null shell. Shell.GetShellOrOtherOpenForm() returns null when no window is open at all -- the bulk-upload and hydrate CLI commands -- and the InvokeRequired test would have thrown a NullReferenceException there, on exactly the old-book-with-a-huge-image case this ticket is about. A null shell now takes the direct path with whatever progress the caller handed us. Note for review: which branch each caller lands in is keyed off shell.InvokeRequired, and that mapping needs a second look before this merges -- see the preflight report on the card. InvokeRequired is true when we are NOT on the UI thread, so as written the dialog is constructed on background threads and the synchronous path runs on the UI thread, which is the opposite of what either symptom needs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
[Claude Opus 5 (1M context) during preflight] Consulted Devin on 2026-08-05 up to commit This was a re-review after a third design landed — a hybrid that reinstates the "Updating Image Files" dialog for callers with no browser progress. Devin found one bug and raised five things to investigate. The bug is the important one: the thread test that picks between the dialog and the direct call is inverted, so the dialog is created on background threads and the silent path runs on the UI thread — the opposite of what either symptom needs. I had reached the same conclusion before Devin filed it, and a local review agent verified the WinForms semantics with an actual probe rather than from memory; that thread carries the caller-by-caller breakdown and two knock-on consequences (a failed shrink still being recorded as migrated, and Also left open, both tied to that same decision: the comments that still describe the previous design (I wrote two of them last run, when they were true), and the lambda parameter that shadows the new The stale PR description was a fair catch and is fixed and resolved — it had fallen a design behind for the second time. Two flags were not posted again because they duplicate existing threads. Devin's "silent shrink for Five informational items were assessed and not mirrored. Two are worth a reviewer's eye and are carried into the report: that the CI ( |
The thread test that chose between our own progress dialog and the caller's IProgress was inverted: InvokeRequired is true when we are NOT on the UI thread, so the dialog was being created on background threads -- the original bug -- while the silent inline path ran on the UI thread, where it froze Bloom with no feedback. Swap the two. Off the UI thread we hand our messages to the progress the caller already owns, since anything that got here from a background thread is running under some progress of its own; on the UI thread there is nothing to borrow and inline work would freeze Bloom, so we put up our own dialog, which does the work on a background worker. Book selection now gets that dialog instead of a silent freeze, and Do Updates of All Books reports into the progress dialog it already has open instead of stacking a second one on top. The shell.Invoke that briefly wrapped the dialog is gone with the swap rather than inverted along with it: the dialog branch is now only reached when InvokeRequired is false, so we are already on the shell's thread and there is nothing to marshal. Also stop recording a book as migrated when the shrink failed. ProgressDialogBackground never reads RunWorkerCompletedEventArgs.Error, so an exception thrown by the work vanished and we carried on to set mediaMaintenanceLevel to 1 -- permanently, because the level is never revisited, so that book's images would stay oversized forever with nothing reported. The work now runs inside a try/catch whose exception is rethrown on the calling thread with ExceptionDispatchInfo, which reports the failure and leaves the level alone, matching what the direct path has always done. The dialog body moves into ShrinkImagesBehindProgressDialog, whose lambda parameter is named dialogProgress rather than shadowing the method's own progress parameter -- the shadowing made it look as though the caller's progress was being honored on that path when it was not. Comments in BookStorage and BookProcessor that still described the interim design (no dialog at all) now describe this one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
[Claude Opus 5 (1M context) during preflight] Consulted Devin on 2026-08-05 up to commit This is the re-review after the developer's decisions were implemented: the thread test was swapped, a failed shrink no longer records the book as migrated, the lambda parameter was renamed, and the comments were brought in line. Devin's own output confirms the main fix landed — one of its informational items reads "The 'inverted thread test' described in the PR body is not present in this head commit." Its bug list still carries the wrong-thread bug and five Investigate flags from earlier commits on this branch; every one of those already has a thread here that was replied to and resolved this run, so they were not posted again:
One genuinely new flag, and a fair one: image-shrink failures now abort the book update instead of being swallowed. That is the intended consequence of the developer's decision not to mark a book migrated when the shrink failed, and the selection path already has a try/catch that enriches and rethrows into Bloom's error reporting — so it surfaces as a reported error naming the book rather than an unhandled crash. Replied with the reasoning and resolved. One informational item was left as accepted rather than fixed, and is written up in the PR description as a known limitation: CI ( |
Make the progress parameter optional, defaulting to null, and substitute a NullProgress when we use it directly. Only the unit tests actually rely on the default: both production call sites already pass something non-null, because Book.EnsureUpToDate substitutes a NullProgress for a null argument. This lets the tests read as what they are -- calls that do not care about progress. Then make the dialog decision match what the doc comment claims. Previously the choice was made purely on the thread, so a caller already on the UI thread got our "Updating Image Files" dialog even when it had a perfectly good visible progress of its own. The single-book "Update Book" command is exactly that case: it runs under ProgressDialogForeground, whose work runs on the UI thread via an Application.Idle handler, so it was getting a second progress dialog stacked inside the one already on screen, and the foreground dialog it had open was written to not at all during the shrink. That was a regression introduced two commits ago when the thread test was swapped -- the commit before it took the direct path here and reported into the existing dialog. So decide on whether there is anywhere real to report, not only on the thread. A NullProgress reports nowhere, so having one is equivalent to having none, and on the UI thread it still earns a dialog; anything else is somewhere real and gets used. Book selection, which is what BL-16646 is really about, is unaffected: it arrives with a NullProgress substituted by EnsureUpToDate, so it keeps its dialog and stays responsive. Exactly one caller changes behavior, and it is the one that was wrong. This also retires the "progress is WebProgressAdapter" test, which was a narrow proxy for this same question -- a WebProgressAdapter is simply one kind of real progress, so it is now covered by the general case rather than named specially. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
[Claude Opus 5 (1M context) during preflight] Consulted Devin on 2026-08-06 up to commit No new bugs, and no new review threads were needed: Devin's bug and Investigate lists are entirely carried-over findings from earlier commits on this branch, and every one of them already has a thread here that was replied to and resolved. Its own informational items confirm as much — one notes that the "inverted thread test" described in an earlier PR body is not present in this head commit. Two informational items were new and worth recording. The one that matters. On the UI thread with a real progress, the shrink now runs inline rather than on a background worker. That is a direct consequence of this commit, which stopped choosing purely on the thread and started asking whether the caller has anywhere real to report. Devin audited the callers and found the only UI-thread caller supplying a non-null progress is The PR description was stale again — it still described the Worth noting what this commit fixes, since it was a regression of my own making: the swap two commits ago moved the single-book "Update Book" command onto the dialog path, so it was stacking an "Updating Image Files" dialog inside the CI ( |
Using the caller's progress also means doing the shrinking synchronously on the caller's thread. That is fine for every caller today, but it puts an unwritten requirement on any future one that is on the UI thread: its progress has to pump messages, or Bloom freezes for the whole shrink. Write that down where someone adding such a caller will see it, along with why the one UI-thread caller we do have is safe -- ProgressDialogForeground runs all of BringBookUpToDate on the UI thread anyway, and its MultiProgress includes an ApplicationDoEventsProgress that pumps on every message. Comment only; no behavior change. Devin raised the consequence as an informational item and the developer chose to record the invariant rather than add machinery for a caller that does not exist yet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…6646) When the shrink runs behind our own "Updating Image Files" dialog, a failure in the background worker was captured and rethrown on the calling thread. That did keep mediaMaintenanceLevel at 0, so the book was not falsely recorded as migrated -- but it also aborted the rest of bringing the book up to date (migrations 2-13 and the Save that follows), for a failure on a single image. Report the failure and let the rest of the update continue: the helper now returns false, and the caller only bumps mediaMaintenanceLevel when the work actually succeeded, so the shrink is retried the next time this book is brought up to date. Retrying is safe -- FixSizeAndTransparencyOfImagesInFolder only resizes a file whose current dimensions differ from the desired ones, so images shrunk on the earlier attempt are skipped rather than re-encoded. Log the whole exception via Logger.WriteError (the idiom already used five times in this file) rather than just its Message. The likeliest failure is a TagLib error on one particular image, where the message alone typically names neither the file nor the inner exception. Also drop the now-unused System.Runtime.ExceptionServices using, and correct the helper's remarks, which still described the rethrow. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…6646) Book.EnsureUpToDate calls MigrateToMediaLevel1ShrinkLargeImages twice: once by way of EnsureUpToDateMemory, and once directly afterwards. That was harmless only because the first call always bumped mediaMaintenanceLevel, so the second returned at the level check. Now that a failed shrink deliberately leaves the level at 0, the second call would scan the folder and start the whole slow shrink over again, put up the "Updating Image Files" dialog a second time, and fail the same way -- the cause (a TagLib error on one image, say) is deterministic. So a book with one bad image would take twice the work and flash the dialog twice, with nothing explained to the user. Remember the failed attempt in a field on the BookStorage, so it happens once per pass. The level still stays at 0, so the shrink is still retried the next time that book is loaded and brought up to date; we just do not do it twice over in one pass. Found by Devin's review of the previous commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
[Claude Opus 5 (1M context) during preflight] Consulted Devin on 2026-08-06 up to commit This was the re-review after the developer changed a failed shrink from rethrowing to logging and carrying on. Devin earned its keep this round: it found one real bug in that change and it is now fixed.
Its remaining four Investigate flags and both listed bugs are carried-over findings from earlier commits on this branch — every one already has a thread here that was replied to and resolved, and Devin now marks both bugs resolved. Eleven further items were informational; none needed action. The full C# suite is green at this commit: 3040 passed, 0 failed, 13 skipped. |
Until now a failed shrink was reported only to the log, and only on the branch that puts up our own dialog; the direct branch let the exception propagate, which abandoned the remaining twelve migrations and the Save, and for "Do Updates of All Books" silently ended the whole batch at the first bad image. Both branches now catch the failure and report it the same way, so a bad image costs that one book's shrink rather than the rest of the operation. The report is a passive NonFatalProblem: the book still works, its pictures are merely left large, and mediaMaintenanceLevel is left at 0 so the shrink is retried the next time the book is brought up to date -- none of which is worth interrupting the user for. NonFatalProblem also logs the whole exception and reports to Sentry. Two details worth recording: The user-visible text is localized (ImageUtils.ShrinkingImagesFailed, in BloomLowPriority.xlf since it is a rarely-seen failure message) rather than an interpolated string, and it no longer leads with the book's folder path -- that belongs in the details that go to the log and Sentry, not in the sentence the user reads. This follows the sibling toast strings in that file. The failure is also written to the caller's progress when it has one, so an operation that is already showing the user a progress box does not appear to have finished cleanly. It deliberately uses WriteWarning, not WriteError: ProgressDialogForeground pops a modal "There was a problem performing that operation" whenever its progress records an error, which would defeat the point of reporting this passively. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
[Claude Opus 5 (1M context) during preflight] Consulted Devin on 2026-08-06 up to commit This was the re-review after the developer settled the open question from the last round: a failed shrink is now caught on both branches and reported passively, so a bad image costs that one book's shrink rather than the rest of the update.
Its other seven Investigate flags and both listed bugs are carried-over findings from earlier commits on this branch — each already has a thread here that was replied to and resolved, and Devin marks both bugs resolved. Twelve further items were informational; none needed action. There are now no open review threads on this PR. The full C# suite is green at this commit: 3031 passed, 0 failed, 13 skipped. That count is lower than the 3040 reported for the previous commit because master has since made |
MigrateToMediaLevel1ShrinkLargeImagesshrinks images larger than 3840x2800 the first time an old book (pre-4.9, still atmediaMaintenanceLevel0) is brought up to date, and showed its own "Updating Image Files" progress dialog while it worked. Several of the operations that bring a book up to date run on a background thread, and WinForms only allows aFormto be created on the UI thread — so on an old book with an oversized image, those paths could hang or leave a stray window.The fix decides where to report based on whether the caller already has somewhere real to report, and whether we could legally put up a dialog at all:
.bloomSourceand spreadsheet import, the publish previews, and the single-book "Update Book" command. Its messages go there, so they land in the window the user is already watching and we add nothing of our own.NullProgress, and we are on the UI thread — most importantly clicking a book to select it. There is nowhere to report, and doing the work inline would freeze Bloom, so we put up our own dialog, which is legal here and which runs the work on a background worker so the app stays responsive.NullProgressif it passed none.MigrateToMediaLevel1ShrinkLargeImagesgained an optionalIProgressparameter for this, passed atBook.cs:1110andBook.cs:1898. Only tests rely on the default — both production call sites pass something non-null, sinceBook.EnsureUpToDatesubstitutes aNullProgressfor a null argument.How a failed shrink is handled
This took several passes to settle, so here is the whole of it in one place.
Originally a failure inside the dialog's background worker vanished —
ProgressDialogBackgroundnever readsRunWorkerCompletedEventArgs.Error— and we setmediaMaintenanceLevelto 1 anyway, permanently, since the level is never revisited. That book's images stayed oversized forever with nothing reported.Now, in both branches:
NonFatalProblem.Report(ModalIf.None, PassiveIf.All, …): a toast, plus the whole exception to the log and Sentry. It is passive on purpose — the book still works, its pictures are merely left large, and we will try again — so it is not worth interrupting the user for.WriteWarning, notWriteError:ProgressDialogForegroundpops a modal "There was a problem performing that operation" whenever its progress records an error, which would defeat the point of reporting passively.ImageUtils.ShrinkingImagesFailed, inBloomLowPriority.xlfas a rarely-seen failure message). The book's folder path goes in the details that reach the log and Sentry, not in the sentence the user reads.mediaMaintenanceLevelis bumped only on success, so the shrink is retried the next time that book is brought up to date. Retrying is safe:FixSizeAndTransparencyOfImagesInFolderonly resizes a file whose current dimensions differ from the desired ones, so images shrunk on an earlier attempt are skipped rather than re-encoded.Book.EnsureUpToDatecalls this migration twice — once by way ofEnsureUpToDateMemoryand once directly afterwards — and the second call was a no-op only because the first had always bumped the level. A flag on theBookStorageremembers the failed attempt, so one pass makes one attempt.The consequence worth a reviewer's eye: a bad image now costs that one book's shrink rather than the rest of the operation. Previously an exception from the direct branch abandoned the remaining twelve migrations and the
Save, and for Do Updates of All Books silently ended the whole batch at the first bad image.Also fixed along the way:
Shell.GetShellOrOtherOpenForm()returns null in the headless CLI, and theInvokeRequiredtest would have thrown aNullReferenceExceptionthere — on exactly the old-book-with-a-huge-image case this ticket is about.Supporting change:
WebProgressAdapternow accepts theIWebSocketProgressinterface rather than the concreteWebSocketProgress, soMakeDerivativeFromBloomSourceFilecan forward the progress it is handed.Notes for the reviewer
The branch has been through several designs; the history shows marshal-the-dialog, delete-the-dialog-entirely, and two shapes of hybrid. Some resolved Devin threads on this PR discuss code that is no longer here, and each is annotated to say so.
One consequence worth a look: when the caller has a real progress and happens to be on the UI thread, the shrink runs inline on that thread rather than on a background worker. Today the only such caller is "Update Book", which runs under
ProgressDialogForegroundand so already does the whole ofBringBookUpToDateon the UI thread — its progress pumpsApplication.DoEventson every message, so the dialog keeps painting and nothing new freezes. It does mean a future UI-thread caller passing a live progress that does not pump would block the UI for the whole shrink. The invariant is written down in a comment at the branch.Known limitations, recorded rather than fixed here, and now tracked as BL-16663:
FixSizeAndTransparencyOfImagesInFolderhas no per-file try/catch, so one unreadable image aborts its loop at the same point on every attempt and the images after it are never shrunk. The retry above therefore cannot get past such a file.false, and one of the two call sites ignores that — so those books are marked migrated with oversized images. The success flag above only catches failures that throw.Both fixes change a method with other callers, so they need their own change and test pass.
Ref: https://issues.bloomlibrary.org/youtrack/issue/BL-16646
Devin review
This change is