Run the automatic update in two requests and say why it failed (#450, 0.7.92)

.gitignore

@@ -601,6 +601,7 @@ pinakes-v*-local.zip.sha256
 # Reinstall / upgrade regression runbook (local-only, not committed)
 reinstall_test.md
 tests/manual-upgrade-real.spec.js
+tests/auto-upgrade-real.spec.js
 docs/reference/start-server.md
 docs/reference/security-audit-report.md
 docs/reference/routes-to-add.md

CHANGELOG.md

@@ -2,6 +2,16 @@
 
 Full version-by-version history for Pinakes. The README shows only the latest release; everything older lives here.
 
+## [0.7.92]
+
+### Fixed
+- **The automatic update no longer fails with "The server returned an invalid response"** ([#450](https://github.com/fabiodalez-dev/Pinakes/issues/450)). It ran the backup, the download from GitHub, the file copy and the migrations in one long request, holding the 30 MB package in memory; on hosting that cuts long requests off behind a proxy, or will not raise PHP's memory limit, the request ended with an error page instead of an answer, while uploading the same package by hand worked. The automatic update now runs as two requests, like a manual one: the server downloads the release and verifies its sha256, then installs it through the same request a manual update ends with. The package is written to disk as it arrives instead of being held in memory, by the single-request update too.
+- **A failed update says why.** A PHP fatal error during an update (memory, a time limit the host enforces) now comes back as a message with PHP's own reason, and an answer that is not JSON at all, such as a proxy's timeout page, is shown with its HTTP status and the start of its text. Before, both read only "invalid response". The update requests also ask for JSON, so an expired session answers in words the page can show.
+- **A download that breaks off is reported as one.** On a slow link the package download can stop partway, for instance on a timeout; the partial file then reached the checksum, and the update failed with "the archive does not match the expected checksum", which reads like a tampered release. It now fails as a download, with the transport's reason. Packages downloaded or uploaded for an install that never ran (a closed tab, a session that expired in between) are also removed after an hour, with the other temporary update folders, instead of staying in storage/tmp.
+- **The install request installs the package its own page prepared.** A download and an upload, in two tabs of the same session, shared one waiting package, and the install request took whichever came last. The page now sends back the id of the package it downloaded or uploaded; a package replaced in another tab is refused, not installed, and the replaced one is deleted. A downloaded package is checked against its sha256 again right before it is installed.
+
+This fix runs from the update after 0.7.92: an installation older than 0.7.92 still updates with its own, older updater. If the automatic update fails there, upload the 0.7.92 package under Admin → Updates → Manual update.
+
 ## [0.7.91]
 
 ### Added

README.md

@@ -41,7 +41,11 @@ Pinakes is a self-hosted, full-featured ILS for schools, municipalities, and pri
 
 Highlights of the latest release are below. The full version-by-version history (v0.7.59 → v0.6.x) lives in **[CHANGELOG.md](CHANGELOG.md)**.
 
-### v0.7.91 — latest
+### v0.7.92 — latest
+
+**The automatic update works where it used to answer "invalid response"** ([#450](https://github.com/fabiodalez-dev/Pinakes/issues/450)). It now downloads and verifies the release in one request and installs it in a second, exactly as a manual update does, and writes the package to disk instead of holding it in memory; a failure shows its real cause (a PHP error, or a proxy's HTTP status and page). An installation older than 0.7.92 updates with its own updater: if the automatic update fails there, upload the 0.7.92 package once under Admin → Updates → Manual update. No migration.
+
+### v0.7.91
 
 **danMARC2, the Danish format, can be imported.** A new DBC preset in Plugins → Z39.50/SRU searches the Danish union catalogue, and any SRU server that answers in danMARC2 works too; before, a danMARC2 record could not be read at all. **Who did what is read from more records:** a translator or editor the record names without a role is found in the title-page statement ("translated by…"), roles are understood in German, French, Spanish, the Scandinavian languages, Dutch and Polish, and a colorist goes to the Colorist picker. Places and editions lose their ISBD brackets ("London [u.a.]" is "London"), and the form gets the ISBN that was searched for. No migration.
 

app/Controllers/UpdateController.php

@@ -151,6 +151,7 @@ public function performUpdate(Request $request, Response $response, mysqli $db):
         }
 
         // Perform the update
+        $this->answerJsonOnFatal();
         $result = $updater->performUpdate($targetVersion);
 
         if ($result['success']) {
@@ -622,10 +623,10 @@ public function uploadUpdate(Request $request, Response $response, mysqli $db):
 
             if ($result['success']) {
                 // Store path in session to avoid leaking filesystem paths to client
-                $_SESSION['manual_update_path'] = $result['path'];
                 return $this->jsonResponse($response, [
                     'success' => true,
-                    'message' => __('Pacchetto caricato con successo')
+                    'message' => __('Pacchetto caricato con successo'),
+                    'package' => $this->holdPendingPackage($updater, (string) $result['path'], null),
                 ]);
             }
 
@@ -643,6 +644,115 @@ public function uploadUpdate(Request $request, Response $response, mysqli $db):
         }
     }
 
+    /**
+     * API: Download the release package for an automatic update.
+     *
+     * First of the two requests of an automatic update: the server fetches the
+     * release from GitHub, verifies its sha256 and keeps it under storage/tmp;
+     * the page then installs it through install-manual, the same request a
+     * manual update ends with. Splitting them keeps each request short enough
+     * for hosting that cuts long requests off behind a proxy (issue #450).
+     */
+    public function downloadUpdatePackage(Request $request, Response $response, mysqli $db): Response
+    {
+        if (($_SESSION['user']['tipo_utente'] ?? '') !== 'admin') {
+            return $this->jsonResponse($response, ['error' => __('Operazione riservata agli amministratori')], 403);
+        }
+
+        $data = (array) $request->getParsedBody();
+        if (!Csrf::validate($data['csrf_token'] ?? '')) {
+            return $this->jsonResponse($response, ['error' => __('Token CSRF non valido')], 403);
+        }
+
+        $version = trim((string) ($data['version'] ?? ''));
+        if ($version === '' || preg_match('/^v?\d+\.\d+\.\d+(?:-[0-9A-Za-z.]+)?$/', $version) !== 1) {
+            return $this->jsonResponse($response, ['error' => __('Versione non specificata')], 400);
+        }
+
+        try {
+            $updater = new Updater($db);
+        } catch (\Throwable $e) {
+            return $this->jsonResponse($response, [
+                'success' => false,
+                'error' => $this->updaterUnavailable($e, 'downloadUpdatePackage'),
+            ], 503);
+        }
+
+        $requirements = $updater->checkRequirements();
+        if (!$requirements['met']) {
+            return $this->jsonResponse($response, [
+                'success' => false,
+                'error' => __('Requisiti di sistema non soddisfatti'),
+                'requirements' => $requirements['requirements']
+            ], 400);
+        }
+
+        $this->answerJsonOnFatal();
+        $result = $updater->downloadPackageForInstall($version);
+        if (!$result['success'] || $result['path'] === null) {
+            return $this->jsonResponse($response, ['success' => false, 'error' => $result['error']], 500);
+        }
+
+        // The path stays on the server, as for an uploaded package
+        return $this->jsonResponse($response, [
+            'success' => true,
+            'message' => __('Pacchetto scaricato e verificato'),
+            'package' => $this->holdPendingPackage($updater, $result['path'], $result['sha256']),
+        ]);
+    }
+
+    /**
+     * Keep a downloaded or uploaded package for the install request, under an
+     * opaque id the page sends back. The install request then installs this
+     * package and no other: a download in one tab and an upload in another
+     * share the session, and without the id the second would take the first's
+     * place unnoticed. A package that is replaced is deleted.
+     */
+    private function holdPendingPackage(Updater $updater, string $path, ?string $sha256): string
+    {
+        $previous = (string) ($_SESSION['manual_update_path'] ?? '');
+        if ($previous !== '' && $previous !== $path) {
+            $updater->discardPendingPackage($previous);
+        }
+        $id = bin2hex(random_bytes(16));
+        $_SESSION['manual_update_path'] = $path;
+        $_SESSION['manual_update_id'] = $id;
+        if ($sha256 !== null) {
+            $_SESSION['manual_update_sha256'] = $sha256;
+        } else {
+            unset($_SESSION['manual_update_sha256']);
+        }
+        return $id;
+    }
+
+    /**
+     * A fatal error during an update (memory, a time limit the host enforces)
+     * would end the request with an HTML error page or nothing at all, and the
+     * page could only say "invalid response". Answer it as JSON with PHP's own
+     * message instead, whenever nothing has been sent yet.
+     */
+    private function answerJsonOnFatal(): void
+    {
+        register_shutdown_function(static function (): void {
+            $error = error_get_last();
+            if ($error === null || !in_array($error['type'], [E_ERROR, E_PARSE, E_CORE_ERROR, E_COMPILE_ERROR], true)) {
+                return;
+            }
+            while (ob_get_level() > 0) {
+                ob_end_clean();
+            }
+            if (headers_sent()) {
+                return;
+            }
+            http_response_code(500);
+            header('Content-Type: application/json; charset=utf-8');
+            echo json_encode([
+                'success' => false,
+                'error' => __('Errore fatale PHP durante l\'aggiornamento') . ': ' . $error['message'],
+            ], JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES);
+        });
+    }
+
     /**
      * API: Install manually uploaded update package
      */
@@ -662,9 +772,22 @@ public function installManualUpdate(Request $request, Response $response, mysqli
             return $this->jsonResponse($response, ['error' => __('Token CSRF non valido')], 403);
         }
 
+        // The page names the package it downloaded or uploaded. When another tab
+        // has replaced it since, refuse without touching the other one, which
+        // its own install request can still take.
+        $pendingId = (string) ($_SESSION['manual_update_id'] ?? '');
+        $requestedId = (string) ($data['package'] ?? '');
+        if (($pendingId !== '' || $requestedId !== '') && !hash_equals($pendingId, $requestedId)) {
+            return $this->jsonResponse($response, [
+                'success' => false,
+                'error' => __('Il pacchetto in attesa è cambiato in un\'altra scheda: ripeti l\'aggiornamento'),
+            ], 409);
+        }
+
         // Retrieve path from session (not from client) to prevent path manipulation
         $tempPath = $_SESSION['manual_update_path'] ?? '';
-        unset($_SESSION['manual_update_path']);
+        $expectedSha256 = (string) ($_SESSION['manual_update_sha256'] ?? '');
+        unset($_SESSION['manual_update_path'], $_SESSION['manual_update_id'], $_SESSION['manual_update_sha256']);
 
         if (empty($tempPath)) {
             return $this->jsonResponse($response, ['error' => __('Path pacchetto non specificato')], 400);
@@ -676,13 +799,25 @@ public function installManualUpdate(Request $request, Response $response, mysqli
         $realTempPath = realpath($tempPath);
         $realStorageTmp = realpath($storageTmp);
 
-        if (!$realTempPath || !$realStorageTmp || !str_starts_with($realTempPath, $realStorageTmp)) {
+        if (!$realTempPath || !$realStorageTmp || !str_starts_with($realTempPath, $realStorageTmp . DIRECTORY_SEPARATOR)) {
             return $this->jsonResponse($response, [
                 'success' => false,
                 'error' => __('Path pacchetto non valido')
             ], 400);
         }
 
+        // A downloaded package is checked again against the digest it was
+        // verified with, right before it is installed
+        if ($expectedSha256 !== '') {
+            $actualSha256 = (string) @hash_file('sha256', $realTempPath . '/update.zip');
+            if (!hash_equals($expectedSha256, $actualSha256)) {
+                return $this->jsonResponse($response, [
+                    'success' => false,
+                    'error' => __('Verifica di integrità fallita: l\'archivio scaricato non corrisponde al checksum atteso.'),
+                ], 400);
+            }
+        }
+
         try {
             $updater = new Updater($db);
         } catch (\Throwable $e) {
@@ -703,6 +838,7 @@ public function installManualUpdate(Request $request, Response $response, mysqli
         }
 
         // Perform the update from uploaded file (use resolved path to prevent TOCTOU)
+        $this->answerJsonOnFatal();
         $result = $updater->performUpdateFromFile($realTempPath);
 
         if ($result['success']) {

app/Routes/web.php

@@ -3593,6 +3593,13 @@
         return $controller->uploadUpdate($request, $response, $db);
     })->add(new CsrfMiddleware())->add(new AdminAuthMiddleware());
 
+    // Automatic update, first request: download and verify the release package
+    $app->post('/admin/updates/download', function ($request, $response) use ($app) {
+        $db = $app->getContainer()->get('db');
+        $controller = new \App\Controllers\UpdateController();
+        return $controller->downloadUpdatePackage($request, $response, $db);
+    })->add(new CsrfMiddleware())->add(new AdminAuthMiddleware());
+
     // Manual update - Install uploaded package
     $app->post('/admin/updates/install-manual', function ($request, $response) use ($app) {
         $db = $app->getContainer()->get('db');