diff --git a/.github/workflows/testing.yaml b/.github/workflows/testing.yaml index ebb98f2d9..7249d1287 100644 --- a/.github/workflows/testing.yaml +++ b/.github/workflows/testing.yaml @@ -40,4 +40,4 @@ jobs: pip install .[testing] - name: Test with pytest - run: pytest -q --no-cov -o log_cli=false + run: pytest -q --no-cov -o log_cli=false --durations=25 diff --git a/CONTEXT.md b/CONTEXT.md index dab555d9a..19fb12762 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -80,23 +80,41 @@ Life Cycle Assessment calculation (inventory + impact). Multi-LCA runs multiple ### Parameter -A named value or formula used to drive exchange amounts or scenarios. Parameter recalculation and Monte Carlo hooks live under `activity_browser/bwutils/parameters/`. +A named value or formula used to drive flow amounts or scenarios. Parameter recalculation and Monte Carlo hooks live under `activity_browser/bwutils/parameters/`. + +### Parameterized flow + +A flow whose amount is given by a formula (optionally using parameters). The Parameters page lists them under **Parameterized Flows**. Recalculation and Monte Carlo use Brightway’s `ParameterizedExchange` index, which is keyed by activity-parameter group. +_Avoid_: parameterized exchange (when meaning this concept or the UI section) ### Uncertainty -Statistical description of exchange (or parameter or CF) variability (`stats_arrays` types). UI preview helpers live under `activity_browser/bwutils/uncertainty.py` and related dialogs. +Statistical description of flow, parameter, or characterization-factor variability. Monte Carlo samples this description. A lognormal uncertainty on a flow may be derived from an applied pedigree. +_Avoid_: treating stored pedigree scores as the sampled input + +### Pedigree + +Five 1–5 data-quality scores on a flow (reliability, completeness, temporal correlation, geographical correlation, further technological correlation). Together with basic uncertainty they are a stored recipe for the **spread** of a lognormal uncertainty (not its central value). The pedigree editor is hidden until the user chooses to use pedigree in that edit; using pedigree applies the recipe (lognormal spread from the scores) on confirm. Inspecting scores in the table cell, or cancelling, does not apply. Unchecking use pedigree restores the distribution and parameters from just before they checked; in-session score edits are discarded and the stored recipe is unchanged. Switching to another uncertainty (or removing it) while use pedigree is on keeps that new choice, turns use pedigree off, and leaves the stored recipe. Clearing pedigree also turns it off and restores the sampled fields, and deletes the stored recipe on confirm. Checking use pedigree again before confirm restores the stored recipe (Clear is undone). When present, the scores are shown with the flow’s uncertainty (not as a separate table column). Not used on parameters or characterization factors. +_Avoid_: pedigree matrix (when meaning these scores — that name is the factor table); data quality indicators (when meaning this pedigree); treating an unapplied pedigree as the current uncertainty + +### Basic uncertainty + +The extra lognormal spread assumed even when all pedigree scores are 1. Part of the pedigree recipe; not sampled on its own. Default is 1 when unset. When the current uncertainty is lognormal and scores exist, it is inferred from that scale and the scores so the recipe matches; if the scale is tighter than the scores alone, inference is not used (default 1). An inferred value is not stored until the user saves the pedigree recipe. +_Avoid_: sample size (the unused sixth ecoinvent pedigree number); treating this as a sixth 1–5 score ### Monte Carlo -Stochastic sampling of uncertain inputs to produce distributions of LCA results. Uncertainties can related to biosphere and technosphere flows, as well as to parameters and characterization factors. See `activity_browser/bwutils/montecarlo.py` and LCA results Monte Carlo UI. +Stochastic sampling of uncertain inputs to produce distributions of LCA results. Uncertainties can relate to biosphere and technosphere flows, as well as to parameters and characterization factors. In **Scenario LCA**, a selected scenario may change flow amounts on the A and B matrices before sampling; those amounts do not relocate database uncertainty. When the same cell is affected by more than one of these, precedence is **scenario amount < uncertainty sampling < parameter sampling** (parameters win on overlap). See `activity_browser/bwutils/montecarlo.py` and LCA results Monte Carlo UI. ### GSA (Global Sensitivity Analysis) -Analysis of how uncertain inputs drive output variance (e.g. SALib-based), based on Monte Carlo snapshots. See `activity_browser/bwutils/sensitivity_analysis.py`. +Analysis of how uncertain inputs drive output variance (e.g. SALib-based), based on Monte Carlo snapshots. In Scenario LCA it uses the last Monte Carlo run’s scenario (not a separate scenario selector). Figure chrome: header shows the last GSA run’s reference flow and impact category; footer shows cutoffs and that Monte Carlo scenario. +_Avoid_: giving GSA its own independent scenario dropdown ### Scenario LCA -An LCA calculation that also considers multiple scenarios for inventory data (based on the superstructure approach). See `activity_browser/bwutils/superstructure/`. +An LCA calculation that also considers multiple scenarios for inventory data (based on the superstructure approach). A chosen scenario replaces selected technosphere and biosphere flow amounts; it does not define new uncertainty. Monte Carlo in scenario mode still samples database (and parameter) uncertainty with precedence scenario amount < uncertainty sampling < parameter sampling. See `activity_browser/bwutils/superstructure/`. +_Avoid_: treating a scenario difference file as an uncertainty distribution ### Scenario name @@ -216,3 +234,4 @@ _Avoid_: required amount, supply amount (use flow amount in UI labels) | “one-shot” / “bw2io native” (LCIA file) | bw2io impact-category file | | “global app settings file” ad hoc | `app.settings` | | “activity graph” / Graph tab (the view) / neighbourhood | Graph explorer | +| “parameterized exchanges” (the concept or Parameters page section) | parameterized flows | diff --git a/activity_browser/app/actions/database/database_duplicate.py b/activity_browser/app/actions/database/database_duplicate.py index 780d4eb42..0fe220ad2 100644 --- a/activity_browser/app/actions/database/database_duplicate.py +++ b/activity_browser/app/actions/database/database_duplicate.py @@ -100,5 +100,5 @@ def run_safely(self, copy_from, copy_to, backend): elif database.backend == "functional_sqlite" and backend == "sqlite": data = bf.convert_functional_sqlite_to_sqlite(data) - new_database.write(data, searchable=metadata.get("searchable")) + new_database.write(data, searchable=metadata.get("searchable"), signal=True) return new_database diff --git a/activity_browser/app/actions/exchange/exchange_modify.py b/activity_browser/app/actions/exchange/exchange_modify.py index ae52995f4..11a6aba8e 100644 --- a/activity_browser/app/actions/exchange/exchange_modify.py +++ b/activity_browser/app/actions/exchange/exchange_modify.py @@ -1,11 +1,8 @@ from bw2data.proxies import ExchangeProxyBase from activity_browser.app.actions.base import ABAction, exception_dialogs -from activity_browser.mod import bw2data as bd -from bw2data.parameters import ActivityParameter from activity_browser.ui.icons import qicons -from ..parameter.parameter_new_automatic import ParameterNewAutomatic from .exchange_formula_remove import ExchangeFormulaRemove @@ -33,25 +30,8 @@ def run(cls, exchange: ExchangeProxyBase, data: dict): exchange.save() if "formula" in data: - cls.parameterize_exchanges(exchange.output.key) + from activity_browser.bwutils.parameters.formula_exchanges import ( + index_parameterized_flows_for_process, + ) - @staticmethod - def parameterize_exchanges(key: tuple) -> None: - """Used whenever a formula is set on an exchange in an activity. - - If no `ActivityParameter` exists for the key, generate one immediately - """ - act = bd.get_activity(key) - query = (ActivityParameter.database == key[0]) & ( - ActivityParameter.code == key[1] - ) - - if not ActivityParameter.select().where(query).count(): - ParameterNewAutomatic.run([key]) - - group = ActivityParameter.get(query).group - - with bd.parameters.db.atomic(): - bd.parameters.remove_exchanges_from_group(group, act) - bd.parameters.add_exchanges_to_group(group, act) - ActivityParameter.recalculate_exchanges(group) + index_parameterized_flows_for_process(exchange.output.key) diff --git a/activity_browser/app/actions/exchange/exchange_uncertainty_modify.py b/activity_browser/app/actions/exchange/exchange_uncertainty_modify.py index 46f92bb58..fcca58af9 100644 --- a/activity_browser/app/actions/exchange/exchange_uncertainty_modify.py +++ b/activity_browser/app/actions/exchange/exchange_uncertainty_modify.py @@ -5,6 +5,7 @@ from activity_browser import app from activity_browser.app.actions.base import ABAction, exception_dialogs from activity_browser.bwutils.commontasks import database_is_locked +from activity_browser.bwutils.uncertainty import uncertainty_initial_from_flow from activity_browser.ui.icons import qicons from activity_browser.ui.dialogs import UncertaintyDialog @@ -26,9 +27,10 @@ def run(exchanges: List[bd.Edge], uncertainty_dict: dict = None): if uncertainty_dict is None: ok, uncertainty_dict = UncertaintyDialog.get_uncertainty_dict( parent=app.main_window, - initial=exchanges[0].uncertainty, + initial=uncertainty_initial_from_flow(exchanges[0]), read_only=read_only, - ) + enable_pedigree=True, + ) if not ok: return @@ -37,5 +39,9 @@ def run(exchanges: List[bd.Edge], uncertainty_dict: dict = None): for exchange in exchanges: for key, value in uncertainty_dict.items(): + if key == "pedigree" and value is None: + if "pedigree" in exchange: + del exchange["pedigree"] + continue exchange[key] = value exchange.save() diff --git a/activity_browser/app/actions/project/project_delete.py b/activity_browser/app/actions/project/project_delete.py index 22f9e2bfd..d69e5a156 100644 --- a/activity_browser/app/actions/project/project_delete.py +++ b/activity_browser/app/actions/project/project_delete.py @@ -1,5 +1,6 @@ import gc import shutil +import sqlite3 from pathlib import Path from qtpy import QtWidgets @@ -95,7 +96,13 @@ def delete_project(name: str, delete_dir: bool): dir_path = _project_directory(name, ds) assert dir_path.is_dir(), "Can't find project directory" _close_sqlite_databases_in(dir_path) - shutil.rmtree(dir_path) + try: + shutil.rmtree(dir_path) + except PermissionError: + # Orphan sqlite handles are often released only after GC on Windows. + gc.collect() + _close_sqlite_databases_in(dir_path) + shutil.rmtree(dir_path) ds.delete_instance() @@ -108,14 +115,32 @@ def _project_directory(name: str, ds: ProjectDataset) -> Path: def _close_sqlite_databases_in(dir_path: Path) -> None: - """Release peewee SQLite handles so Windows can delete the project directory.""" + """Release SQLite handles so Windows can delete the project directory. + + Peewee ``db.close()`` only closes the *current* thread's connection + (``thread_safe=True``). Raw ``sqlite3.connect`` and leftovers after a + worker thread exits are closed via a connection scan. + """ + directory = Path(dir_path) for _, substitutable_db in config.sqlite3_databases: try: - if Path(substitutable_db._filepath).is_relative_to(dir_path): + if Path(substitutable_db._filepath).is_relative_to(directory): if not substitutable_db.db.is_closed(): substitutable_db.db.close() except Exception: pass + + for obj in gc.get_objects(): + if not isinstance(obj, sqlite3.Connection): + continue + try: + for _, _, filename in obj.execute("PRAGMA database_list"): + if filename and Path(filename).is_relative_to(directory): + obj.close() + break + except Exception: + # Other-thread connections raise ProgrammingError; ignore. + pass gc.collect() diff --git a/activity_browser/app/actions/project/project_migrate25.py b/activity_browser/app/actions/project/project_migrate25.py index 4f74ef19c..7a1a7fcda 100644 --- a/activity_browser/app/actions/project/project_migrate25.py +++ b/activity_browser/app/actions/project/project_migrate25.py @@ -140,7 +140,7 @@ def update_database_activity_types(cls, db_name: str): ds["type"] = "processwithreferenceproduct" if write: - database.write(raw) + database.write(raw, signal=True) @staticmethod def activity_is_processwithreferenceproduct(ds: dict) -> bool: diff --git a/activity_browser/app/pages/activity_details/activity_details.py b/activity_browser/app/pages/activity_details/activity_details.py index 536fa34a1..1941a5632 100644 --- a/activity_browser/app/pages/activity_details/activity_details.py +++ b/activity_browser/app/pages/activity_details/activity_details.py @@ -119,7 +119,7 @@ def on_node_deleted(self, node): Args: node: The node that was deleted. """ - if node.id == self.activity.id: + if self.activity is None or node.id == self.activity.id: self.deleteLater() def on_database_deleted(self, name): @@ -129,7 +129,7 @@ def on_database_deleted(self, name): Args: name: The name of the database that was deleted. """ - if name == self.activity["database"]: + if self.activity is None or name == self.activity["database"]: self.deleteLater() def syncLater(self): @@ -145,9 +145,13 @@ def sync(self): self.activity = refresh_node_or_none(self.activity) if self.activity is None: - # Activity was already deleted + # Activity / database already gone — close rather than sync stale tabs + self.deleteLater() return + # Keep child tabs aligned with the refreshed proxy + self.parameters_tab.activity = self.activity + # Update the tab name to be the activity name self.setWindowTitle(self.activity["name"]) diff --git a/activity_browser/app/pages/activity_details/activity_header.py b/activity_browser/app/pages/activity_details/activity_header.py index adf06f77b..e8eca43fc 100644 --- a/activity_browser/app/pages/activity_details/activity_header.py +++ b/activity_browser/app/pages/activity_details/activity_header.py @@ -246,6 +246,8 @@ class ActivityLocation(QtWidgets.QLineEdit): _WIDTH_PAD_PX = 14 _MIN_WIDTH_PX = 32 + _EDIT_MIN_WIDTH_PX = 180 + _EDIT_MAX_WIDTH_PX = 420 def __init__(self, header: ActivityHeader): """ @@ -264,18 +266,53 @@ def __init__(self, header: ActivityHeader): self.setFixedHeight(fm.height() + 4) self.setSizePolicy(QtWidgets.QSizePolicy.Policy.Fixed, QtWidgets.QSizePolicy.Policy.Fixed) self.textChanged.connect(self._adjust_width_to_text) - self._adjust_width_to_text() self.editingFinished.connect(self.change_location) - locations = set(app.metadata.dataframe.get("location", ["GLO"])) - completer = QtWidgets.QCompleter(locations, self) + locations = {str(loc) for loc in set(app.metadata.dataframe.get("location", ["GLO"])) if loc == loc and loc} + self._edit_min_width = self._width_for_locations(locations) + completer = QtWidgets.QCompleter(sorted(locations), self) + completer.setCaseSensitivity(QtCore.Qt.CaseSensitivity.CaseInsensitive) + completer.popup().setMinimumWidth(self._edit_min_width) self.setCompleter(completer) + self._adjust_width_to_text() + + def _width_for_locations(self, locations: set[str]) -> int: + fm = QtGui.QFontMetrics(self.font()) + widest = max((fm.horizontalAdvance(loc) for loc in locations), default=0) + return min( + max(widest + self._WIDTH_PAD_PX, self._EDIT_MIN_WIDTH_PX), + self._EDIT_MAX_WIDTH_PX, + ) def _adjust_width_to_text(self) -> None: fm = QtGui.QFontMetrics(self.font()) t = self.text() text_w = fm.horizontalAdvance(t) if t else fm.horizontalAdvance(" ") - self.setFixedWidth(max(text_w + self._WIDTH_PAD_PX, self._MIN_WIDTH_PX)) + if self.hasFocus(): + self.setFixedWidth(max(text_w + self._WIDTH_PAD_PX, self._edit_min_width)) + else: + self.setFixedWidth(max(text_w + self._WIDTH_PAD_PX, self._MIN_WIDTH_PX)) + if self.hasFocus() and not t.strip(): + self._popup_all_locations() + + def _popup_all_locations(self) -> None: + """Show the full location list when the field is blank.""" + completer = self.completer() + if completer is None: + return + completer.setCompletionPrefix("") + completer.complete() + + def focusInEvent(self, event: QtGui.QFocusEvent) -> None: + super().focusInEvent(event) + self._adjust_width_to_text() + if not self.text().strip(): + # Defer so the popup opens after focus is fully established. + QtCore.QTimer.singleShot(0, self._popup_all_locations) + + def focusOutEvent(self, event: QtGui.QFocusEvent) -> None: + super().focusOutEvent(event) + self._adjust_width_to_text() def change_location(self): """ diff --git a/activity_browser/app/pages/activity_details/exchanges_tab.py b/activity_browser/app/pages/activity_details/exchanges_tab.py index e92d6e7c9..9106c734e 100644 --- a/activity_browser/app/pages/activity_details/exchanges_tab.py +++ b/activity_browser/app/pages/activity_details/exchanges_tab.py @@ -15,7 +15,7 @@ is_node_product_or_waste, is_node_biosphere, parameters_in_scope, is_node_product, is_node_waste, get_exchange_type, classify_dragged_nodes) -from activity_browser.bwutils.uncertainty import uncertainty_cell_summary +from activity_browser.bwutils.uncertainty import uncertainty_cell_summary, uncertainty_initial_from_flow from activity_browser.ui import widgets, icons, delegates, core @@ -185,7 +185,10 @@ def build_df(self, exchanges) -> pd.DataFrame: # Create a DataFrame from the exchanges exc_df = pd.DataFrame(exchanges, columns=["amount", "input", "formula", "comment", "type"]) - exc_df["uncertainty"] = [uncertainty_cell_summary(x.uncertainty) for x in exchanges] + exc_df["uncertainty"] = [ + uncertainty_cell_summary(x.uncertainty, pedigree=x.get("pedigree")) + for x in exchanges + ] act_df = app.metadata.get_metadata(exc_df["input"].unique(), cols).rename(columns={"type": "_producer_type"}) # Merge the exchanges DataFrame with the metadata DataFrame @@ -662,19 +665,16 @@ def mimeData(self, indices: list[QtCore.QModelIndex]) -> core.ABMimeData: return data def uncertainty_editor_initial(self, index: QtCore.QModelIndex) -> dict: - initial = super().uncertainty_editor_initial(index) - if initial: - return initial row = self.row(index) if row is None: return {} ex = row.get("_exchange") if ex is None: return {} - u = getattr(ex, "uncertainty", None) # retrieve the existing uncertainty dict - if isinstance(u, dict): - return dict(u) - return {} + return uncertainty_initial_from_flow(ex) + + def uncertainty_editor_enable_pedigree(self, index: QtCore.QModelIndex) -> bool: + return self.column_name(index) == "uncertainty" def uncertainty_editor_read_only(self, index: QtCore.QModelIndex) -> bool: if self.column_name(index) != "uncertainty": diff --git a/activity_browser/app/pages/activity_details/parameters_tab.py b/activity_browser/app/pages/activity_details/parameters_tab.py index 220230986..594943d34 100644 --- a/activity_browser/app/pages/activity_details/parameters_tab.py +++ b/activity_browser/app/pages/activity_details/parameters_tab.py @@ -9,7 +9,14 @@ from activity_browser import app from activity_browser.ui import widgets, icons, delegates, core -from activity_browser.bwutils.commontasks import refresh_node, refresh_parameter, parameters_in_scope, database_is_locked, node_group +from activity_browser.bwutils.commontasks import ( + refresh_node, + refresh_node_or_none, + refresh_parameter, + parameters_in_scope, + database_is_locked, + node_group, +) from activity_browser.bwutils.uncertainty import uncertainty_cell_summary from activity_browser.bwutils.utils import Parameter @@ -68,6 +75,10 @@ def sync(self): """ logger.log("SYNC", f"{self.__class__.__name__}: {id(self)}") + self.activity = refresh_node_or_none(self.activity) + if self.activity is None: + return + df = self.build_df() self.model.set_dataframe(df, group=["_param_type", "_scope"]) self.view.expandAll() @@ -105,7 +116,7 @@ def build_df(self) -> pd.DataFrame: row = self._parameter_to_row(param, db_name, db_name) translated.append(row) - if not database_is_locked(db_name): + if db_name in bd.databases and not database_is_locked(db_name): translated.append({ "name": "New parameter...", "_scope": db_name, @@ -123,7 +134,9 @@ def build_df(self) -> pd.DataFrame: row = self._parameter_to_row(param, f"Group: {group_name}", param.database) translated.append(row) - if not database_is_locked(self.activity["database"]): + if self.activity["database"] in bd.databases and not database_is_locked( + self.activity["database"] + ): translated.append({ "name": "New parameter...", "_scope": f"Group: {group_name}", diff --git a/activity_browser/app/pages/calculation_setup/scenario_section.py b/activity_browser/app/pages/calculation_setup/scenario_section.py index 75f6ec202..b6ebd71bb 100644 --- a/activity_browser/app/pages/calculation_setup/scenario_section.py +++ b/activity_browser/app/pages/calculation_setup/scenario_section.py @@ -602,9 +602,17 @@ def __init__(self, index: int, parent=None): self.index = index self.file_path = None self.sheet_index = None + self.csv_separator = ";" self.scenario_name = QtWidgets.QLabel("", self) self.load_btn = QtWidgets.QPushButton(icons.qicons.import_db, "Load") self.load_btn.setToolTip("Load (new) data for this scenario table") + refresh_icon = self.style().standardIcon( + QtWidgets.QStyle.StandardPixmap.SP_BrowserReload + ) + self.reload_btn = QtWidgets.QToolButton(self) + self.reload_btn.setIcon(refresh_icon) + self.reload_btn.setToolTip("Reload the same scenario file from disk") + self.reload_btn.setEnabled(False) self.remove_btn = QtWidgets.QPushButton(icons.qicons.delete, "Delete") self.remove_btn.setToolTip("Remove this scenario table") self.view = ScenarioImportView(self) @@ -617,6 +625,7 @@ def __init__(self, index: int, parent=None): row = QtWidgets.QHBoxLayout() row.addWidget(self.scenario_name) row.addWidget(self.load_btn) + row.addWidget(self.reload_btn) row.addStretch(1) row.addWidget(self.remove_btn) @@ -628,11 +637,42 @@ def __init__(self, index: int, parent=None): def connect_signals(self): self.load_btn.clicked.connect(self.load_action) + self.reload_btn.clicked.connect(self.reload_action) parent = self.parent() if parent and isinstance(parent, ScenarioSection): self.remove_btn.clicked.connect(lambda: parent.remove_table(self.index)) self.remove_btn.clicked.connect(parent.can_add_table) + def _update_filename_label(self, path: Path | None, *, ok: bool) -> None: + """Show the scenario file name; style red when the last load failed.""" + if path is None: + self.scenario_name.setText("") + self.scenario_name.setToolTip("") + self.scenario_name.setStyleSheet("") + return + path = Path(path) + self.scenario_name.setText(path.name) + self.scenario_name.setToolTip(str(path)) + if ok: + self.scenario_name.setStyleSheet("") + else: + self.scenario_name.setStyleSheet( + "color: #c62828; font-weight: bold;" + ) + + def _warn_load_failed(self, path: Path, *, title: str = "Could not load scenario file") -> None: + while QtWidgets.QApplication.overrideCursor() is not None: + QtWidgets.QApplication.restoreOverrideCursor() + QtWidgets.QMessageBox.warning( + self, + title, + f"Could not load scenario file:\n{path}\n\n" + "No usable scenario data was found. If Excel has the file open, " + "save it and try again, or close Excel so Activity Browser can " + "read it. Otherwise check that the file is a valid flow- or " + "parameter-scenario file.", + ) + def load_action(self) -> None: dialog = ExcelReadDialog(self) if dialog.exec_() != ExcelReadDialog.DialogCode.Accepted: @@ -646,9 +686,24 @@ def load_action(self) -> None: ok = self.load_from_path( path, sheet_index=idx, separator=separator or ";" ) - if not ok: - return - self._parent.save_button(True) + if ok: + self._parent.save_button(True) + finally: + while QtWidgets.QApplication.overrideCursor() is not None: + QtWidgets.QApplication.restoreOverrideCursor() + + def reload_action(self) -> None: + if not self.file_path: + return + QtWidgets.QApplication.setOverrideCursor(Qt.WaitCursor) + try: + ok = self.load_from_path( + self.file_path, + sheet_index=self.sheet_index, + separator=self.csv_separator, + ) + if ok: + self._parent.save_button(True) finally: while QtWidgets.QApplication.overrideCursor() is not None: QtWidgets.QApplication.restoreOverrideCursor() @@ -658,7 +713,15 @@ def _looks_like_flow_sdf(df: pd.DataFrame) -> bool: return ( df is not None and not df.empty - and len(df.columns.intersection(ss.SUPERSTRUCTURE)) >= 12 + and ss.is_flow_sdf_headers(df.columns) + ) + + @staticmethod + def _looks_like_broken_flow_sdf(df: pd.DataFrame) -> bool: + return ( + df is not None + and not df.empty + and ss.is_partial_flow_sdf_headers(df.columns) ) @staticmethod @@ -669,6 +732,24 @@ def _looks_like_parameter_scenarios(df: pd.DataFrame) -> bool: and len(df.columns.intersection({"Name", "Group"})) == 2 ) + def _warn_header_mismatch(self, path: Path, columns) -> None: + missing = ss.missing_superstructure_columns(columns) + while QtWidgets.QApplication.overrideCursor() is not None: + QtWidgets.QApplication.restoreOverrideCursor() + missing_html = ", ".join(f'"{m}"' for m in missing) + msg = ( + "

The scenario file header does not match the expected flow-scenario " + f"columns.

Missing or misspelled required column(s):
{missing_html}" + "

Expected headers:
" + + ss.edit_superstructure_for_string(sep=", ", fhighlight='"') + + "

" + f"

File:
{path}

" + ) + critical = ss.ABPopup.abCritical( + "Invalid scenario file header", msg, QtWidgets.QPushButton("Cancel") + ) + critical.exec_() + def _read_excel_scenario_df( self, path: Path, sheet_index: int | None ) -> pd.DataFrame: @@ -688,8 +769,10 @@ def _read_excel_scenario_df( for idx in candidates: df = ss.import_from_excel(path, idx) - if self._looks_like_flow_sdf(df) or self._looks_like_parameter_scenarios( - df + if ( + self._looks_like_flow_sdf(df) + or self._looks_like_broken_flow_sdf(df) + or self._looks_like_parameter_scenarios(df) ): self.sheet_index = idx return df @@ -709,19 +792,38 @@ def load_from_path( file_type_suffix = path.suffix.lower() logger.info("Loading Scenario file. This may take a while for large files") self.file_path = path + self.csv_separator = separator - if file_type_suffix == ".feather": - df = ss.ABFeatherImporter.read_file(path) - self.sheet_index = None - elif file_type_suffix.startswith(".xls"): - df = self._read_excel_scenario_df(path, sheet_index) - else: - df = ss.ABCSVImporter.read_file(path, separator=separator) - self.sheet_index = None + try: + if file_type_suffix == ".feather": + df = ss.ABFeatherImporter.read_file(path) + self.sheet_index = None + elif file_type_suffix.startswith(".xls"): + df = self._read_excel_scenario_df(path, sheet_index) + else: + df = ss.ABCSVImporter.read_file(path, separator=separator) + self.sheet_index = None + except Exception: + logger.exception("Failed to read scenario file: {}", path) + df = pd.DataFrame() if df is None or getattr(df, "empty", False): + logger.warning("Scenario file read returned no usable data: {}", path) + self._update_filename_label(path, ok=False) + self.reload_btn.setEnabled( + self.file_path is not None and Path(self.file_path).is_file() + ) + if not quiet: + self._warn_load_failed(path) + return False + + if self._looks_like_broken_flow_sdf(df): + self._update_filename_label(path, ok=False) + self.reload_btn.setEnabled( + self.file_path is not None and Path(self.file_path).is_file() + ) if not quiet: - logger.warning("Scenario file read returned no usable data: {}", path) + self._warn_header_mismatch(path, df.columns) return False if self._looks_like_flow_sdf(df): @@ -734,8 +836,14 @@ def load_from_path( df["Group"] = df["Group"].astype(str) self.sync_superstructure(ss.parameters_to_sdf(df), combine=combine) else: + self._update_filename_label(path, ok=False) + self.reload_btn.setEnabled( + self.file_path is not None and Path(self.file_path).is_file() + ) if quiet: return False + while QtWidgets.QApplication.overrideCursor() is not None: + QtWidgets.QApplication.restoreOverrideCursor() msg = ( "The Activity-Browser is attempting to import a scenario file.

During the attempted import" " another file type was detected. Please check the file type of the attempted import, if it is" @@ -755,9 +863,10 @@ def load_from_path( critical.exec_() return False - self.scenario_name.setText(path.name) - self.scenario_name.setToolTip(path.name) - return not self.scenario_df.empty + ok = not self.scenario_df.empty + self._update_filename_label(path, ok=ok) + self.reload_btn.setEnabled(True) + return ok def sync_superstructure(self, df: pd.DataFrame, combine: bool = True) -> None: """synchronizes the contents of either a single, or multiple scenario files to create a single scenario diff --git a/activity_browser/app/pages/lca_results/LCA_results.py b/activity_browser/app/pages/lca_results/LCA_results.py index 0f10e459a..1e1faf785 100644 --- a/activity_browser/app/pages/lca_results/LCA_results.py +++ b/activity_browser/app/pages/lca_results/LCA_results.py @@ -39,6 +39,7 @@ lcia_compare_mode_from_label, ) from activity_browser.bwutils.sensitivity_analysis import GlobalSensitivityAnalysis +from activity_browser.bwutils.superstructure.mlca import SuperstructureMLCA from activity_browser.mod.bw2analyzer import ABContributionAnalysis from activity_browser.ui import widgets @@ -66,6 +67,28 @@ ca = ABContributionAnalysis() +def _format_mc_run_footer(summary: dict) -> str: + parts = [] + if summary.get("scenario"): + parts.append(f"Scenario: {summary['scenario']}") + parts.append(f"Iterations: {summary['iterations']}") + parts.append(f"Random seed: {summary['seed']}") + includes = summary.get("includes") or {} + flags = "".join( + letter + for key, letter in ( + ("technosphere", "T"), + ("biosphere", "B"), + ("cf", "C"), + ("parameters", "P"), + ) + if includes.get(key) + ) + if flags: + parts.append(f"Included: {flags}") + return " · ".join(parts) + + # Special namedtuple for the LCAResults TabWidget. Tabs = namedtuple( "tabs", ("inventory", "results", "ef", "process", "contribution_tree", "sankey", "mc", "gsa") @@ -1300,7 +1323,7 @@ def update_dataframe(self, *args, **kwargs): raise NotImplementedError def update_table(self): - super().update_table(self.df, unit=self.unit) + super().update_table(self.df, unit=self.unit, tab=self) def update_plot(self): """Update the plot.""" @@ -1803,17 +1826,24 @@ class MonteCarloTab(NewAnalysisTab): def __init__(self, parent=None): super(MonteCarloTab, self).__init__(parent) self.parent: LCAResultsSubTab = parent + self.df = None + self._results_stale = False self.explain_text = """ -

Monte Carlo Analyses

-

Monte Carlo simulations generate stochastic data samples using existing data defined parameter - distributions for generating the expected distribution for the reference flows.

-

More simply, within the LCA model the user may define certain uncertainty distributions for some - (or all) parameters. Monte Carlo analysis uses these defined uncertainty distributions with a stochastic - generator to sample from these distributions. This results in a "posterior" (or final) probability - distribution, expressing the expected variance, for the reference flows.

-

More - information can be found here

+

Monte Carlo simulation draws random samples from uncertainty + distributions defined on technosphere flows, biosphere flows, characterization + factors, and/or parameters, then recalculates LCA scores for each iteration.

+

Iterations — number of stochastic draws (more iterations give smoother + distributions but take longer).

+

Random seed — optional integer for reproducible samples. Leave empty + to let Brightway choose a seed; the value actually used is shown in the footer + after a run.

+

Include uncertainty for — which layers resample each iteration: + Technosphere, Biosphere, Characterization Factors, Parameters. Uncertainty + distributions come from the database (not from the static flow amount).

+

In scenario mode, the selected scenario sets flow amounts first; then + the same sampling rules apply. Precedence on a cell: scenario amount, then + uncertainty sampling, then parameter sampling (parameters win on overlap).

""" self.add_tab_header( "Monte Carlo Simulation", @@ -1839,6 +1869,13 @@ def __init__(self, parent=None): self.add_MC_ui_elements() + self.stale_label = QtWidgets.QLabel( + "Scenario changed — click Run to update Monte Carlo results." + ) + self.stale_label.setWordWrap(True) + self.stale_label.hide() + self.layout.addWidget(self.stale_label) + self.table = LCAResultsTable() mc_basename = lca_export_basename(self.parent.cs_name, "Monte Carlo") self.table.table_name = mc_basename @@ -1846,6 +1883,12 @@ def __init__(self, parent=None): self.plot.plot_name = mc_basename self.add_tab_body_with_placeholder() + + self.run_info_label = QtWidgets.QLabel("") + self.run_info_label.setWordWrap(True) + self.run_info_label.hide() + self.layout.addWidget(self.run_info_label) + self.export_widget = self.add_tab_footer( has_plot=True, has_table=True, wrapped=True ) @@ -1979,9 +2022,29 @@ def calculate_mc_lca(self): "parameters": self.include_parameters.isChecked(), } + calc_kwargs = dict( + iterations=iterations, + seed=seed, + **includes, + ) + if self.has_scenarios and isinstance(self.parent.mlca, SuperstructureMLCA): + scenario_name = self._selected_scenario_name() + if scenario_name is None: + QtWidgets.QMessageBox.warning( + self, + "Warning", + "Select a scenario before running Monte Carlo.", + ) + return + calc_kwargs["scenario_overlay"] = self.parent.mlca.scenario_overlay( + scenario_name + ) + QtWidgets.QApplication.setOverrideCursor(QtCore.Qt.WaitCursor) try: - self.parent.mc.calculate(iterations=iterations, seed=seed, **includes) + self.parent.mc.calculate(**calc_kwargs) + self._results_stale = False + self.stale_label.hide() app.signals.monte_carlo_finished.emit() self.update_mc() except ( @@ -2052,10 +2115,38 @@ def configure_scenario(self): super().configure_scenario() self.scenario_label.setVisible(self.has_scenarios) + def _selected_scenario_name(self) -> Optional[str]: + labels = self.get_scenario_labels() + if not labels: + return None + index = max(self.scenario_box.currentIndex(), 0) + if 0 <= index < len(labels): + return labels[index] + return labels[0] + + def _update_stale_state(self) -> None: + if self.df is None or not self.has_scenarios: + self._results_stale = False + self.stale_label.hide() + return + last = getattr(self.parent.mc, "last_run_scenario", None) + selected = self._selected_scenario_name() + self._results_stale = bool(last and selected and last != selected) + self.stale_label.setVisible(self._results_stale) + + def _update_run_info_footer(self) -> None: + summary = self.parent.mc.last_run_summary + if not summary: + self.run_info_label.hide() + return + self.run_info_label.setText(_format_mc_run_footer(summary)) + self.run_info_label.show() + @QtCore.Slot(int, name="mcScenarioIndexChanged") def _on_scenario_index_changed(self, index: int) -> None: - """Scenario only affects export names after MC has been run.""" - if self.df is not None: + """Keep prior results until Run; mark stale when selection differs from last run.""" + self._update_stale_state() + if not self._results_stale and self.df is not None: self.update_mc() def update_tab(self): @@ -2085,13 +2176,13 @@ def update_mc(self, cs_name=None): self.update_plot(method=method) self.space_check() fields = [self.parent.cs_name, "Monte Carlo", method] - if self.has_scenarios: - scenario_index = max(self.scenario_box.currentIndex(), 0) - scenario_names = self.get_scenario_labels() - if scenario_names and 0 <= scenario_index < len(scenario_names): - fields.append(scenario_names[scenario_index]) + last_scenario = getattr(self.parent.mc, "last_run_scenario", None) + if last_scenario: + fields.append(last_scenario) filename = lca_export_basename(*fields) self.plot.plot_name, self.table.table_name = filename, filename + self._update_run_info_footer() + self._update_stale_state() def update_plot(self, method): super().update_plot(self.df, method=method) @@ -2238,10 +2329,14 @@ def add_GSA_ui_elements(self): self.label_monte_carlo_first = QtWidgets.QLabel( "You need to run a Monte Carlo Simulation first." ) + self.mc_scenario_label = QtWidgets.QLabel("") + self.mc_scenario_label.setWordWrap(True) self.layout.addWidget(self.label_monte_carlo_first) + self.layout.addWidget(self.mc_scenario_label) self.layout.addWidget(self.widget_settings) self.widget_settings.hide() + self.mc_scenario_label.hide() def update_tab(self): self.update_combobox( @@ -2257,6 +2352,14 @@ def monte_carlo_finished(self): self.button_run.setEnabled(True) self.widget_settings.show() self.label_monte_carlo_first.hide() + scenario = getattr(self.parent.mc, "last_run_scenario", None) + if scenario: + self.mc_scenario_label.setText( + f"GSA uses the last Monte Carlo run (scenario: {scenario})." + ) + self.mc_scenario_label.show() + else: + self.mc_scenario_label.hide() def calculate_gsa(self): act_number = self.combobox_fu.currentIndex() diff --git a/activity_browser/app/pages/lca_results/combobox_utils.py b/activity_browser/app/pages/lca_results/combobox_utils.py index 1b7ad5b0c..ae6c33a87 100644 --- a/activity_browser/app/pages/lca_results/combobox_utils.py +++ b/activity_browser/app/pages/lca_results/combobox_utils.py @@ -14,7 +14,7 @@ "set_combobox_index", "update_combobox", "scenario_labels", - "configure_scenario_widgets", + "configure_selection_comboboxes", ] @@ -58,17 +58,22 @@ def scenario_labels(parent) -> list[str]: return list(getattr(mlca, "scenario_names", []) or []) -def configure_scenario_widgets( +def configure_selection_comboboxes( *, - has_scenarios: bool, + parent, + fu_box: QtWidgets.QComboBox, + method_box: QtWidgets.QComboBox, scenario_box: QtWidgets.QComboBox, scenario_label: QtWidgets.QLabel, - parent, -) -> list[str]: - """Show/hide scenario controls and refresh the scenario combo.""" + has_scenarios: bool, +) -> None: + """Populate RF / IC / scenario combos from the calculated MLCA (active CS rows).""" + mlca = getattr(parent, "mlca", None) + fu_labels = list(getattr(mlca, "fu_labels", {}).values()) if mlca else [] + method_labels = [str(m) for m in getattr(mlca, "methods", [])] if mlca else [] + update_combobox(fu_box, fu_labels) + update_combobox(method_box, method_labels) scenario_box.setVisible(has_scenarios) scenario_label.setVisible(has_scenarios) - labels = scenario_labels(parent) if has_scenarios else [] if has_scenarios: - update_combobox(scenario_box, labels) - return labels + update_combobox(scenario_box, scenario_labels(parent)) diff --git a/activity_browser/app/pages/lca_results/contribution_tree_tab.py b/activity_browser/app/pages/lca_results/contribution_tree_tab.py index b14a26dab..62bcea30d 100644 --- a/activity_browser/app/pages/lca_results/contribution_tree_tab.py +++ b/activity_browser/app/pages/lca_results/contribution_tree_tab.py @@ -45,7 +45,7 @@ from activity_browser.ui.icons import qicons from activity_browser.ui.selection_history import IndexSelectionHistory -from .combobox_utils import configure_scenario_widgets, scenario_labels, update_combobox +from .combobox_utils import configure_selection_comboboxes from .contribution_tree_model import ( BAR_COLUMNS, COL_CUMULATIVE, @@ -479,36 +479,19 @@ def update_tab(self) -> None: self._reload_from_state(expanded_uids=uids, model_uids=model_uids) def configure_scenario(self) -> None: - configure_scenario_widgets( - has_scenarios=self.has_scenarios, + configure_selection_comboboxes( + parent=self.parent, + fu_box=self.fu_cb, + method_box=self.method_cb, scenario_box=self.scenario_cb, scenario_label=self.scenario_label, - parent=self.parent, + has_scenarios=self.has_scenarios, ) - update_combobox = staticmethod(update_combobox) - def _update_calculation_setup(self, cs_name: str = None) -> None: for w in (self.fu_cb, self.method_cb, self.scenario_cb): w.blockSignals(True) - - cs = cs_name or (self.parent.cs_name if self.parent else None) - if cs is None: - for w in (self.fu_cb, self.method_cb, self.scenario_cb): - w.blockSignals(False) - return - - setup = bd.calculation_setups.get(cs, {}) - fu_acts = [ - list({bd.get_activity(k): v for k, v in fu.items()}.keys())[0] - for fu in setup.get("inv", []) - ] - self.fu_cb.clear() - self.fu_cb.addItems([f"{repr(a)} | {a._data.get('database')}" for a in fu_acts]) - self.method_cb.clear() - self.method_cb.addItems([repr(m) for m in setup.get("ia", [])]) self.configure_scenario() - for w in (self.fu_cb, self.method_cb, self.scenario_cb): w.blockSignals(False) self._seed_selection_history() @@ -533,10 +516,9 @@ def _selection_inputs(self, key: tuple): """Resolve demand dict and method tuple for a cache key.""" fu_idx, method_idx, scenario_idx, _cutoff_pct = key - cs = self.parent.cs_name - setup = bd.calculation_setups[cs] - demand_raw = setup["inv"][fu_idx] - method = setup["ia"][method_idx] + mlca = self.parent.mlca + demand_raw = mlca.func_units[fu_idx] + method = mlca.methods[method_idx] demand = {bd.get_activity(k).id: v for k, v in demand_raw.items()} return demand, method, scenario_idx diff --git a/activity_browser/app/pages/lca_results/plots.py b/activity_browser/app/pages/lca_results/plots.py index bdfc39f75..a22c4b18f 100644 --- a/activity_browser/app/pages/lca_results/plots.py +++ b/activity_browser/app/pages/lca_results/plots.py @@ -422,11 +422,16 @@ def plot(self, df: pd.DataFrame, method: tuple): continue color = self.series_color(j) label = legend_display[j] if j < len(legend_display) else legend_full[j] - self.ax.hist( - vals, density=True, alpha=0.5, - label=label, color=color, - ) - self.ax.axvline(float(np.mean(vals)), color=color) + if vals.size == 0: + continue + if np.allclose(vals, vals[0]): + self.ax.axvline(float(vals[0]), color=color, label=label, linewidth=2) + else: + self.ax.hist( + vals, density=True, alpha=0.5, + label=label, color=color, + ) + self.ax.axvline(float(np.mean(vals)), color=color) self.ax.set_xlabel(unit) self.ax.set_ylabel("Probability") diff --git a/activity_browser/app/pages/lca_results/sankey_navigator_tab.py b/activity_browser/app/pages/lca_results/sankey_navigator_tab.py index 22e67f88b..da238a94d 100644 --- a/activity_browser/app/pages/lca_results/sankey_navigator_tab.py +++ b/activity_browser/app/pages/lca_results/sankey_navigator_tab.py @@ -60,7 +60,7 @@ SmallComboBox, ) -from .combobox_utils import configure_scenario_widgets, scenario_labels, update_combobox +from .combobox_utils import configure_selection_comboboxes, scenario_labels # Runaway guard per NNEV hop, not the Adjust policy stop. SANKEY_MAX_CALC = 1000 @@ -207,9 +207,6 @@ def __init__(self, cs_name, parent=None): self.cs = cs_name self.plot_name = lca_export_basename(cs_name, "Sankey") self.has_sankey = False - self.func_units = [] - self.methods = [] - self.scenarios = [] self.graph = Graph() # Additional Qt objects @@ -407,39 +404,40 @@ def configure_scenario(self): """Determine if scenario Qt widgets are visible or not and retrieve scenario labels for the selection drop-down box. """ - configure_scenario_widgets( - has_scenarios=self.has_scenarios, + configure_selection_comboboxes( + parent=self.parent, + fu_box=self.func_unit_cb, + method_box=self.method_cb, scenario_box=self.scenario_cb, scenario_label=self.scenario_label, - parent=self.parent, + has_scenarios=self.has_scenarios, ) self.scenarios = self.get_scenario_labels() - update_combobox = staticmethod(update_combobox) + @property + def func_units(self): + mlca = getattr(self.parent, "mlca", None) + if mlca is None: + return [] + return [ + {bd.get_activity(k): v for k, v in fu.items()} + for fu in mlca.func_units + ] + + @property + def methods(self): + mlca = getattr(self.parent, "mlca", None) + return list(mlca.methods) if mlca else [] def update_calculation_setup(self, cs_name=None) -> None: """Update Calculation Setup, reference flows and impact categories, and dropdown menus.""" - # block signals self.func_unit_cb.blockSignals(True) self.method_cb.blockSignals(True) self.scenario_cb.blockSignals(True) self.cs = cs_name or self.cs - self.func_units = [ - {bd.get_activity(k): v for k, v in fu.items()} - for fu in bd.calculation_setups[self.cs]["inv"] - ] - self.methods = bd.calculation_setups[self.cs]["ia"] - self.func_unit_cb.clear() - fu_acts = [list(fu.keys())[0] for fu in self.func_units] - self.func_unit_cb.addItems( - [f"{repr(a)} | {a._data.get('database')}" for a in fu_acts] - ) self.configure_scenario() - self.method_cb.clear() - self.method_cb.addItems([repr(m) for m in self.methods]) - # unblock signals self.func_unit_cb.blockSignals(False) self.method_cb.blockSignals(False) self.scenario_cb.blockSignals(False) diff --git a/activity_browser/app/pages/lca_results/tables.py b/activity_browser/app/pages/lca_results/tables.py index 2cde9f51d..3143f0e28 100644 --- a/activity_browser/app/pages/lca_results/tables.py +++ b/activity_browser/app/pages/lca_results/tables.py @@ -17,6 +17,7 @@ from activity_browser.ui.icons import qicons from activity_browser.ui import delegates from activity_browser.bwutils import filesystem +from activity_browser.bwutils.contribution_labels import apply_contribution_column_labels from .dialogs import FilterManagerDialog, SimpleFilterDialog @@ -998,12 +999,17 @@ def write_filters(self, filters: dict) -> None: class ContributionModel(PandasModel): - def sync(self, df, unit="% of range"): - + def sync(self, df, unit="% of range", tab=None): + df = df.copy() if "unit" in df.columns: # overwrite the unit col when showing relative results (except 3 'total' and 'rest' rows) df["unit"] = [""] * 3 + [unit] * (len(df) - 3) + table = self.parent() + if tab is None and table is not None: + tab = getattr(table, "tab", None) + df = apply_contribution_column_labels(df, tab) + # drop any rows where all numbers are 0 self._dataframe = df.loc[~(df.select_dtypes(include=np.number) == 0).all(axis=1)] self.updated.emit() @@ -1012,5 +1018,7 @@ def sync(self, df, unit="% of range"): class ContributionTable(ABDataFrameView): def __init__(self, parent=None): super().__init__(parent) + # QObject parent may later be the plot/table body widget; keep the tab. + self.tab = parent self.model = ContributionModel(parent=self) self.model.updated.connect(self.update_proxy_model) diff --git a/activity_browser/app/pages/parameters/parameterized_exchanges_section.py b/activity_browser/app/pages/parameters/parameterized_exchanges_section.py index 914b5c1e3..bbf1d66fc 100644 --- a/activity_browser/app/pages/parameters/parameterized_exchanges_section.py +++ b/activity_browser/app/pages/parameters/parameterized_exchanges_section.py @@ -4,29 +4,44 @@ from qtpy.QtCore import Qt import pandas as pd -import bw2data as bd -from bw2data.parameters import ParameterizedExchange -from bw2data.backends import ExchangeDataset from activity_browser import app from activity_browser.ui import widgets, icons, delegates, core from activity_browser.bwutils.commontasks import ( database_is_locked, - exchange_consumer_parts, - exchange_label, - exchange_product_name, ) -from activity_browser.bwutils.uncertainty import uncertainty_cell_summary +from activity_browser.bwutils.parameters.formula_exchanges import indexed_parameterized_flows +from activity_browser.bwutils.uncertainty import ( + uncertainty_cell_summary, + uncertainty_initial_from_flow, +) from activity_browser.bwutils.utils import Parameter +def _meta_row(meta, key): + if meta is None: + return {} + try: + row = meta.loc[key] + return row.iloc[0] if isinstance(row, pd.DataFrame) else row + except Exception: + return {} + + +def _cell(meta, field, fallback=None): + value = meta.get(field) if hasattr(meta, "get") else None + if value is None or (isinstance(value, float) and pd.isna(value)): + return fallback + return value + + class ParameterizedExchangesSection(QtWidgets.QWidget): """ - A widget section that displays all parameterized exchanges in the current project. + A widget section that displays all parameterized flows in the current project. Attributes: - model (ParameterizedExchangesModel): The model containing the data for the exchanges. - view (ParameterizedExchangesView): The view displaying the exchanges. + model (ParameterizedExchangesModel): The model containing the data for the flows. + view (ParameterizedExchangesView): The view displaying the flows. """ def __init__(self, parent=None): @@ -85,51 +100,57 @@ def sync(self): self.model.set_dataframe(df) def build_exchanges_df(self) -> pd.DataFrame: - """ - Builds a DataFrame from all parameterized exchanges in the project. + """Build a DataFrame from Brightway's parameterized-flow index.""" + try: + flows = list(indexed_parameterized_flows()) + except Exception: + logger.opt(exception=True).debug("Parameterized flow index unreadable") + flows = [] + + keys = [k for flow in flows for k in (flow["input_key"], flow["output_key"])] + try: + meta = ( + app.metadata.get_metadata( + keys, ["name", "unit", "product", "location", "database"] + ) + if keys + else None + ) + if meta is not None and not meta.empty: + meta = meta.sort_index() + except Exception: + logger.opt(exception=True).debug("Metadata lookup failed for parameterized flows") + meta = None - Returns: - pd.DataFrame: The DataFrame containing the parameterized exchanges data. - """ translated = [] - - # Get all parameterized exchanges - for param_exc in ParameterizedExchange.select(): - try: - exchange = bd.Edge(document=ExchangeDataset.get_by_id(param_exc.exchange)) - - input_key = exchange.get("input") - output_key = exchange.get("output") - - input_meta = app.metadata.get_metadata( - [input_key], ["name", "unit", "location", "database", "product"] - ).iloc[0] - - product = exchange_product_name(input_key) - process, location, database = exchange_consumer_parts(output_key) - - u = getattr(exchange, "uncertainty", None) - if not isinstance(u, dict): - u = {} - row = { - "amount": exchange.get("amount"), - "unit": input_meta.get("unit"), - "product": product, - "process": process, - "location": location, - "database": database, - "formula": exchange.get("formula"), - "comment": exchange.get("comment"), - "uncertainty": uncertainty_cell_summary(u), - "_exchange_label": exchange_label(input_key, output_key, include_database=True), - "_exchange": exchange, - "_output_key": output_key, - "_input_key": input_key, - } - translated.append(row) - except Exception: - # Skip if exchange can't be loaded - continue + for flow in flows: + input_key = flow["input_key"] + output_key = flow["output_key"] + in_meta = _meta_row(meta, input_key) + out_meta = _meta_row(meta, output_key) + product = _cell(in_meta, "product") or _cell(in_meta, "name") + process = _cell(out_meta, "name") + location = _cell(out_meta, "location") + database = _cell(out_meta, "database", output_key[0]) + loc_bit = f" [{location}]" if location else "" + translated.append({ + "amount": flow["amount"], + "unit": _cell(in_meta, "unit"), + "product": product, + "process": process, + "location": location, + "database": database, + "formula": flow["formula"], + "comment": flow["comment"], + "uncertainty": uncertainty_cell_summary( + flow["uncertainty"], + pedigree=(flow["exchange"].get("pedigree") if flow.get("exchange") else None), + ), + "_exchange_label": f"{product} | {process}{loc_bit} ({database})", + "_exchange": flow["exchange"], + "_output_key": output_key, + "_input_key": input_key, + }) columns = [ "amount", "unit", "product", "process", "location", "database", @@ -259,19 +280,16 @@ def setData(self, index: QtCore.QModelIndex, value, role: int = Qt.ItemDataRole. return False def uncertainty_editor_initial(self, index: QtCore.QModelIndex) -> dict: - initial = super().uncertainty_editor_initial(index) - if initial: - return initial row = self.row(index) if row is None: return {} ex = row.get("_exchange") if ex is None: return {} - u = getattr(ex, "uncertainty", None) - if isinstance(u, dict): - return dict(u) - return {} + return uncertainty_initial_from_flow(ex) + + def uncertainty_editor_enable_pedigree(self, index: QtCore.QModelIndex) -> bool: + return self.column_name(index) == "uncertainty" def uncertainty_editor_read_only(self, index: QtCore.QModelIndex) -> bool: if self.column_name(index) != "uncertainty": diff --git a/activity_browser/app/pages/parameters/parameters.py b/activity_browser/app/pages/parameters/parameters.py index 4da380837..4795d223a 100644 --- a/activity_browser/app/pages/parameters/parameters.py +++ b/activity_browser/app/pages/parameters/parameters.py @@ -8,11 +8,11 @@ class ParametersPage(widgets.ABAbstractPage): """ - A widget that displays all parameters and parameterized exchanges in the current project. + A widget that displays all parameters and parameterized flows in the current project. This page shows: - Parameters section: A tree view of parameters organized by scope - - Parameterized exchanges section: A table of exchanges with formulas + - Parameterized Flows section: A table of flows with formulas in the Brightway index """ basePage = True title = "Parameters" @@ -55,7 +55,7 @@ def build_layout(self): exchanges_widget = QtWidgets.QWidget() exchanges_layout = QtWidgets.QVBoxLayout(exchanges_widget) exchanges_layout.setContentsMargins(0, 0, 0, 0) - exchanges_label = widgets.ABLabel.demiBold(" Parameterized Exchanges") + exchanges_label = widgets.ABLabel.demiBold(" Parameterized Flows") exchanges_layout.addWidget(exchanges_label) exchanges_layout.addWidget(widgets.ABHLine(self)) exchanges_layout.addWidget(self.parameterized_exchanges_section) diff --git a/activity_browser/app/signalling.py b/activity_browser/app/signalling.py index 16fd0d744..18fea4f51 100644 --- a/activity_browser/app/signalling.py +++ b/activity_browser/app/signalling.py @@ -241,7 +241,12 @@ def _on_database_reset(self, sender, name): def _on_database_write(self, sender, name): from bw2data import Database + from activity_browser.bwutils.parameters.formula_exchanges import ( + rebuild_parameterized_flow_index, + ) + t = time() + rebuild_parameterized_flow_index(name) self.database.written.emit(Database(name)) logger.log("SIGNAL", f"Database: written: {time() - t:.2f} seconds") diff --git a/activity_browser/bwutils/README.md b/activity_browser/bwutils/README.md index 01fdb6da4..c3b794b38 100644 --- a/activity_browser/bwutils/README.md +++ b/activity_browser/bwutils/README.md @@ -14,7 +14,7 @@ This module provides a collection of generic methods and utilities that wrap and - **`io/`** - Import/export operations for data interchange - **`metadata/`** - Metadata loading and caching for quick access - **`searchengine/`** - Fuzzy search functionality for dataframes -- **`superstructure/`** - Superstructure scenario analysis tools +- **`superstructure/`** - Superstructure scenario analysis tools (includes ``scenario_overlay`` for matrix overlays shared with Monte Carlo) ## Key Files @@ -24,10 +24,10 @@ This module provides a collection of generic methods and utilities that wrap and - **`exporters.py`** - Export functionality for databases and activities - **`importers.py`** - Import functionality for various LCA data formats - **`filesystem.py`** - File system operations for Brightway2 data directories +- **`montecarlo/`** - Monte Carlo simulation via ``MultiLCA``; matrix_utils patch; scenario dataframe fallback; stores per-iteration matrix snapshots for GSA - **`parameters/`** - Parameter recalculation, Monte Carlo matrix hook, functional_sqlite identity (see `parameters/README.md`) -- **`montecarlo.py`** - Monte Carlo simulation; stores per-iteration matrix snapshots for GSA - **`multilca.py`** - Multi-functional LCA calculation utilities -- **`pedigree.py`** - Pedigree matrix uncertainty handling +- **`pedigree.py`** - Pedigree matrix conversion; infer basic uncertainty; in-dialog pedigree session; resolve pedigree edits on a flow - **`sensitivity_analysis.py`** - SALib delta GSA on ``MonteCarloLCA``; ``df_final`` columns in ``GSA_COLUMNS``; runnable via ``if __name__ == "__main__"`` - **`settings.py`** - Settings specific to bwutils operations - **`strategies.py`** - Import strategies and data transformation functions diff --git a/activity_browser/bwutils/commontasks.py b/activity_browser/bwutils/commontasks.py index 4ff0404ad..1ff9aa38f 100644 --- a/activity_browser/bwutils/commontasks.py +++ b/activity_browser/bwutils/commontasks.py @@ -12,6 +12,7 @@ import bw2data as bd from bw2data.parameters import ParameterBase, ProjectParameter, DatabaseParameter, ActivityParameter, Group from bw2data.errors import UnknownObject +from peewee import OperationalError from functools import lru_cache @@ -203,9 +204,13 @@ def get_database_metadata(name): return d def database_is_locked(name: str) -> bool: - """Returns True if the database is locked.""" - if not name in bd.databases: - raise KeyError("Not an existing database:", name) + """Returns True if the database is locked (or missing / unknown). + + Missing databases are treated as locked so UI sync during delete does not + raise when leftover parameter rows still reference a just-removed database. + """ + if not name or name not in bd.databases: + return True return bd.databases[name].get("read_only", True) def database_is_legacy(name: str) -> bool: @@ -442,39 +447,43 @@ def parameters_in_scope( ) -> dict[str, Parameter]: if (not node and not parameter) or (node and parameter): raise ValueError("Supply either node or parameter") - if node: - node = refresh_node(node) - database = node["database"] - group = node_group(node) - else: # if parameter - parameter = refresh_parameter(parameter) - group = parameter.group - if group == "project": - database = None - elif group in bd.databases: - database = group + try: + if node: + node = refresh_node(node) + database = node["database"] + group = node_group(node) else: - database = ActivityParameter.get_or_none(group=group).database + parameter = refresh_parameter(parameter) + group = parameter.group + if group == "project": + database = None + elif group in bd.databases: + database = group + else: + database = ActivityParameter.get_or_none(group=group).database - data = OrderedDict() + data = OrderedDict() - for name, param in ProjectParameter.load().items(): - data[name] = Parameter(name, "project", param["amount"], param, "project") + for name, param in ProjectParameter.load().items(): + data[name] = Parameter(name, "project", param["amount"], param, "project") - for name, param in DatabaseParameter.load(database).items(): - if name in data: - del data[name] # the variable is overwritten in the scope chain - data[name] = Parameter(name, database, param["amount"], param, "database") + for name, param in DatabaseParameter.load(database).items(): + if name in data: + del data[name] + data[name] = Parameter(name, database, param["amount"], param, "database") - group_deps = Group.get_or_none(name=group).order + [group] if group else [] + group_deps = Group.get_or_none(name=group).order + [group] if group else [] - for dep in group_deps: - for name, param in ActivityParameter.load(dep).items(): - if name in data: - del data[name] # the variable is overwritten in the scope chain - data[name] = Parameter(name, dep, param.get("amount"), param, "activity") + for dep in group_deps: + for name, param in ActivityParameter.load(dep).items(): + if name in data: + del data[name] + data[name] = Parameter(name, dep, param.get("amount"), param, "activity") - return data + return data + except OperationalError: + logger.debug("Parameter scope unavailable (database locked)") + return {} def node_group(node: tuple | int | bd.Node) -> str | None: diff --git a/activity_browser/bwutils/contribution_labels.py b/activity_browser/bwutils/contribution_labels.py index 53285d618..730a6c4d6 100644 --- a/activity_browser/bwutils/contribution_labels.py +++ b/activity_browser/bwutils/contribution_labels.py @@ -1,4 +1,4 @@ -"""Axis and legend labels for LCA contribution plots.""" +"""Axis, legend, and table labels for LCA contributions.""" from __future__ import annotations @@ -90,6 +90,15 @@ def contribution_column_labels(tab, column_keys: list) -> list[str]: ] +def apply_contribution_column_labels(df: pd.DataFrame, tab) -> pd.DataFrame: + """Replace setup-index comparison columns (0, 1, …) with MLCA display labels.""" + keys = [c for c in df.columns if setup_index(c) is not None] + if not keys: + return df + labels = contribution_column_labels(tab, keys) + return df.rename(columns=dict(zip(keys, labels))) + + def _fallback_column_label(col) -> str: if isinstance(col, tuple): return get_method_label(col) diff --git a/activity_browser/bwutils/lcia_overview.py b/activity_browser/bwutils/lcia_overview.py index 9ead655b3..10e530ebe 100644 --- a/activity_browser/bwutils/lcia_overview.py +++ b/activity_browser/bwutils/lcia_overview.py @@ -9,7 +9,9 @@ import numpy as np import pandas as pd -from activity_browser.bwutils.commontasks import unit_of_method +import bw2data as bd + +from activity_browser.bwutils.commontasks import reference_flow_parts, unit_of_method if TYPE_CHECKING: from activity_browser.bwutils.multilca import MLCA @@ -53,6 +55,10 @@ def lcia_compare_labels_for_modes(modes: list[LCIACompareMode]) -> list[str]: RELATIVE_Y_LABEL = "% of max |impact|" +# Trailing identity columns on the LCA scores table (plot still uses labels). +RF_TABLE_COLUMNS = ("amount", "unit", "product", "process", "location", "database") +SCORE_TABLE_COLUMNS = ("index", "series", "absolute", "relative", "score unit") + @dataclass class LCIAOverviewPanel: @@ -145,7 +151,7 @@ def _grouped_matrix( relative: bool, group_units: dict[str, str], y_label: str, -) -> tuple[np.ndarray, np.ndarray, list[str], list[str]]: +) -> tuple[np.ndarray, np.ndarray, np.ndarray, list[str], list[str]]: if groups_along_dim0: abs_values = absolute.astype(float) group_labels = dim0_labels @@ -155,39 +161,81 @@ def _grouped_matrix( group_labels = dim1_labels series_labels = dim0_labels - values = ( - abs_values - if not relative - else _normalize_grouped_matrix(abs_values, relative=relative) - ) - return values, abs_values, group_labels, series_labels + rel_values = _normalize_grouped_matrix(abs_values, relative=True) + values = rel_values if relative else abs_values + return values, abs_values, rel_values, group_labels, series_labels + + +def _reference_flow_table_rows(mlca: MLCA | SuperstructureMLCA) -> list[dict]: + """One identity row per calculation-setup reference flow (stable FU index).""" + rows: list[dict] = [] + empty = {col: "" for col in RF_TABLE_COLUMNS} + for fu in mlca.func_units: + if not fu: + rows.append(dict(empty)) + continue + key = next(iter(fu)) + amount = next(iter(fu.values())) + product = process = location = unit = database = "" + try: + act = bd.get_activity(key) + product, process, location, database = reference_flow_parts(act) + unit = str(act.get("unit") or "") + except Exception: + if isinstance(key, tuple) and key: + database = str(key[0]) + rows.append( + { + "amount": amount, + "unit": unit, + "product": product, + "process": process, + "location": location, + "database": database, + } + ) + return rows def _table_from_matrix( - values: np.ndarray, abs_values: np.ndarray, + rel_values: np.ndarray, group_labels: list[str], series_labels: list[str], group_units: dict[str, str], *, panel: str | None = None, series_units: dict[str, str] | None = None, + fu_rows: list[dict] | None = None, + fu_on: str = "group", ) -> pd.DataFrame: rows = [] for g_idx, group in enumerate(group_labels): for s_idx, series in enumerate(series_labels): - unit = group_units.get(group, "") or (series_units or {}).get(series, "") + score_unit = group_units.get(group, "") or (series_units or {}).get(series, "") row = { "index": group, "series": series, - "value": float(values[g_idx, s_idx]), "absolute": float(abs_values[g_idx, s_idx]), - "unit": unit, + "relative": float(rel_values[g_idx, s_idx]), + "score unit": score_unit, } + if fu_rows: + fu_idx = g_idx if fu_on == "group" else s_idx + if 0 <= fu_idx < len(fu_rows): + row.update(fu_rows[fu_idx]) if panel is not None: row["impact category"] = panel rows.append(row) - return pd.DataFrame(rows) + df = pd.DataFrame(rows) + if df.empty: + return df + leading = [c for c in SCORE_TABLE_COLUMNS if c in df.columns] + if "impact category" in df.columns: + leading.append("impact category") + trailing = [c for c in RF_TABLE_COLUMNS if c in df.columns] + rest = [c for c in df.columns if c not in leading and c not in trailing] + return df.loc[:, leading + rest + trailing] def _method_group_units(method_labels: list[str], mlca: MLCA) -> dict[str, str]: @@ -202,30 +250,28 @@ def _flows_x_methods_matrix( *, relative: bool, flip_groups: bool, -) -> tuple[np.ndarray, np.ndarray, list[str], list[str], dict[str, str], str]: +) -> tuple[np.ndarray, np.ndarray, np.ndarray, list[str], list[str], dict[str, str], str]: """FU × IC matrix; relative scores normalized per impact category.""" abs_matrix = absolute.astype(float) - if relative: - cell_values = normalize_lcia_matrix(abs_matrix, relative=relative) - else: - cell_values = abs_matrix + cell_relative = normalize_lcia_matrix(abs_matrix, relative=True) if flip_groups: - values = cell_values abs_values = abs_matrix + rel_values = cell_relative group_labels = fu_labels series_labels = method_labels group_units = {g: "" for g in fu_labels} y_label = RELATIVE_Y_LABEL if relative else "impact" else: - values = cell_values.T abs_values = abs_matrix.T + rel_values = cell_relative.T group_labels = method_labels series_labels = fu_labels group_units = method_units y_label = RELATIVE_Y_LABEL if relative else "impact" - return values, abs_values, group_labels, series_labels, group_units, y_label + values = rel_values if relative else abs_values + return values, abs_values, rel_values, group_labels, series_labels, group_units, y_label def build_lcia_overview( @@ -242,13 +288,14 @@ def build_lcia_overview( fu_labels = list(mlca.fu_labels.values()) method_labels = list(mlca.method_labels.values()) method_units = _method_group_units(method_labels, mlca) + fu_rows = _reference_flow_table_rows(mlca) if compare == LCIACompareMode.REFERENCE_FLOWS: absolute = lcia_scores_array(mlca, scenario_index)[:, [method_index]] unit = unit_of_method(mlca.methods[method_index]) group_units = {g: unit for g in fu_labels} y_label = unit if not relative else RELATIVE_Y_LABEL - values, abs_values, group_labels, series_labels = _grouped_matrix( + values, abs_values, rel_values, group_labels, series_labels = _grouped_matrix( absolute, fu_labels, [method_labels[method_index]], @@ -263,6 +310,7 @@ def build_lcia_overview( ( values, abs_values, + rel_values, group_labels, series_labels, group_units, @@ -288,7 +336,7 @@ def build_lcia_overview( else {g: unit for g in scenario_labels} ) y_label = unit if not relative else RELATIVE_Y_LABEL - values, abs_values, group_labels, series_labels = _grouped_matrix( + values, abs_values, rel_values, group_labels, series_labels = _grouped_matrix( absolute, fu_labels, scenario_labels, @@ -313,7 +361,7 @@ def build_lcia_overview( else {g: unit for g in scenario_labels} ) y_label = unit if not relative else RELATIVE_Y_LABEL - p_values, p_abs, p_groups, p_series = _grouped_matrix( + p_values, p_abs, p_rel, p_groups, p_series = _grouped_matrix( absolute, fu_labels, scenario_labels, @@ -335,12 +383,14 @@ def build_lcia_overview( ) table_parts.append( _table_from_matrix( - p_values, p_abs, + p_rel, p_groups, p_series, group_units, panel=method_label, + fu_rows=fu_rows, + fu_on="group" if not flip_groups else "series", ) ) return LCIAOverviewData( @@ -358,18 +408,27 @@ def build_lcia_overview( else: raise ValueError(f"Unknown compare mode: {compare}") + if compare == LCIACompareMode.REFERENCE_FLOWS: + fu_on = "group" + elif compare == LCIACompareMode.FLOWS_X_METHODS: + fu_on = "group" if flip_groups else "series" + else: + fu_on = "group" if not flip_groups else "series" + series_units = ( method_units if compare == LCIACompareMode.FLOWS_X_METHODS and flip_groups else None ) table_df = _table_from_matrix( - values, abs_values, + rel_values, group_labels, series_labels, group_units, series_units=series_units, + fu_rows=fu_rows, + fu_on=fu_on, ) return LCIAOverviewData( diff --git a/activity_browser/bwutils/montecarlo/__init__.py b/activity_browser/bwutils/montecarlo/__init__.py new file mode 100644 index 000000000..628aa950f --- /dev/null +++ b/activity_browser/bwutils/montecarlo/__init__.py @@ -0,0 +1,14 @@ +"""Monte Carlo LCA engine and helpers.""" + +from activity_browser.bwutils.montecarlo.engine import MonteCarloLCA, perform_MonteCarlo_LCA +from activity_browser.bwutils.montecarlo.matrix_patch import apply_matrix_utils_mc_patch +from activity_browser.bwutils.montecarlo.scenarios import build_overlay_from_df + +apply_matrix_utils_mc_patch() # TODO: remove once bw2data 4.8 includes the matrix_utils fix + +__all__ = [ + "MonteCarloLCA", + "perform_MonteCarlo_LCA", + "apply_matrix_utils_mc_patch", + "build_overlay_from_df", +] diff --git a/activity_browser/bwutils/montecarlo.py b/activity_browser/bwutils/montecarlo/engine.py similarity index 58% rename from activity_browser/bwutils/montecarlo.py rename to activity_browser/bwutils/montecarlo/engine.py index 920647a4c..dffc899bf 100644 --- a/activity_browser/bwutils/montecarlo.py +++ b/activity_browser/bwutils/montecarlo/engine.py @@ -10,7 +10,7 @@ """ from collections import defaultdict from time import time -from typing import Optional +from typing import Callable, Optional from loguru import logger import bw2data as bd @@ -18,14 +18,72 @@ import numpy as np import pandas as pd -from activity_browser.bwutils.multilca import _load_cs -from activity_browser.bwutils.montecarlo_matrix_utils_patch import apply_matrix_utils_mc_patch +from activity_browser.bwutils.multilca import _load_cs, databases_for_fu_keys +from activity_browser.bwutils.montecarlo.scenarios import build_overlay_from_df from activity_browser.bwutils.parameters import ( MonteCarloParameterManager, bind_parameter_hook, ) +from activity_browser.bwutils.superstructure.scenario_overlay import ( + ScenarioOverlay, + apply_scenario_overlay, +) + + +def _bind_selective_ab_iteration( + lca: bc.MultiLCA, + *, + sample_technosphere: bool, + sample_biosphere: bool, + skip_ab_after_scenario_pinned: Callable[[], bool], +) -> None: + """Skip technosphere/biosphere ``next()`` when that layer is not MC-resampled. + + After the scenario overlay is pinned once (first ``after_matrix_iteration``), + A/B iterators are skipped so Brightway does not rebuild those matrices from + the database datapackage on every iteration. + """ + if sample_technosphere and sample_biosphere: + return + + skip_tech = not sample_technosphere + skip_bio = not sample_biosphere + scenario_pinned = False + + def __next__(self) -> None: + nonlocal scenario_pinned + skip_first_iteration = getattr(self, "keep_first_iteration_flag", False) + + if not skip_first_iteration: + self._delete_solver_state() -apply_matrix_utils_mc_patch() # TODO: remove this patch as soon as bw2data 4.8 is released (it should be fixed there, but test) + defer_ab = skip_ab_after_scenario_pinned() and scenario_pinned + for matrix in self.matrix_labels: + if defer_ab and matrix == "technosphere_mm" and skip_tech: + continue + if defer_ab and matrix == "biosphere_mm" and skip_bio: + continue + if hasattr(self, matrix): + next(getattr(self, matrix)) + + for matrix_dict in self.matrix_list_labels: + if hasattr(self, matrix_dict): + next(getattr(self, matrix_dict)) + + if hasattr(self, "after_matrix_iteration"): + self.after_matrix_iteration() + if skip_ab_after_scenario_pinned(): + scenario_pinned = True + + if bc.PYPARDISO: + self.technosphere_matrix = self.technosphere_matrix.tocsr() + + if skip_first_iteration: + delattr(self, "keep_first_iteration_flag") + + self._calculation() + + lca.__next__ = __next__.__get__(lca, type(lca)) class MonteCarloLCA(object): @@ -43,6 +101,8 @@ def __init__(self, cs_name, cs: dict | None = None): self.include_biosphere = True self.include_cfs = False self.include_parameters = False + self.last_run_scenario = None + self.last_run_includes = None _load_cs(self, self.cs["inv"], self.cs["ia"]) self.method_index = {m: i for i, m in enumerate(self.methods)} @@ -87,8 +147,16 @@ def construct_lca( seed_override=seed_override, ) - def calculate(self, iterations: int = 10, seed: Optional[int] = None, **kwargs): - """Run Monte Carlo LCA with optional technosphere, biosphere, CF, and parameter uncertainty.""" + def calculate( + self, + iterations: int = 10, + seed: Optional[int] = None, + scenario_overlay: Optional[ScenarioOverlay] = None, + scenario_df: Optional[pd.DataFrame] = None, + scenario: Optional[str | int] = None, + **kwargs, + ): + """Run Monte Carlo LCA with optional technosphere, biosphere, CF, parameter, and scenario amounts.""" start = time() self.iterations = iterations self.seed = seed or bc.utils.get_seed() @@ -96,6 +164,11 @@ def calculate(self, iterations: int = 10, seed: Optional[int] = None, **kwargs): self.include_biosphere = kwargs.get("biosphere", True) self.include_cfs = kwargs.get("cf", True) self.include_parameters = kwargs.get("parameters", False) + self.last_run_scenario = None + self.last_run_includes = None + + overlay = scenario_overlay + scenario_name = overlay.name if overlay is not None else None # Parameter amounts are applied in after_matrix_iteration (after matrix draws). if self.include_parameters: @@ -110,13 +183,69 @@ def calculate(self, iterations: int = 10, seed: Optional[int] = None, **kwargs): seed_override=self.seed, ) + self.lca.lci() + self.lca.lcia() + + if overlay is None and scenario_df is not None: + overlay = build_overlay_from_df( + self.lca, + scenario_df, + databases_for_fu_keys(self.fu_activity_keys), + scenario, + ) + if overlay is not None: + scenario_name = overlay.name + + before_parameters = None + if overlay is not None: + scenario_initialized = False + + def apply_scenario(lca: bc.MultiLCA) -> None: + nonlocal scenario_initialized + if not scenario_initialized: + apply_scenario_overlay( + lca, + overlay, + include_technosphere=self.include_technosphere, + include_biosphere=self.include_biosphere, + ) + scenario_initialized = True + elif not self.include_technosphere and not self.include_biosphere: + # A/B ``next()`` is skipped after the first iteration; re-pin because + # ``lci_calculation`` can refresh matrices from unchanged mm state. + apply_scenario_overlay( + lca, + overlay, + include_technosphere=False, + include_biosphere=False, + ) + elif self.include_technosphere or self.include_biosphere: + apply_scenario_overlay( + lca, + overlay, + include_technosphere=self.include_technosphere, + include_biosphere=self.include_biosphere, + repin_only=True, + ) + + before_parameters = apply_scenario + + _bind_selective_ab_iteration( + self.lca, + sample_technosphere=self.include_technosphere, + sample_biosphere=self.include_biosphere, + skip_ab_after_scenario_pinned=lambda: overlay is not None, + ) + self.parameter_mc_manager = None if self.include_parameters: self.parameter_mc_manager = MonteCarloParameterManager(seed=self.seed) - bind_parameter_hook(self.lca, self) - self.lca.lci() - self.lca.lcia() + if before_parameters is not None or self.include_parameters: + bind_parameter_hook( + self.lca, self, before_parameters=before_parameters + ) + # Always sample iteration 0; do not reuse the deterministic baseline matrices. self.lca.keep_first_iteration_flag = False @@ -153,6 +282,26 @@ def calculate(self, iterations: int = 10, seed: Optional[int] = None, **kwargs): f"{len(self.methods)} methods in {np.round(time() - start, 2)} seconds." ) + self.last_run_scenario = scenario_name + self.last_run_includes = { + "technosphere": self.include_technosphere, + "biosphere": self.include_biosphere, + "cf": self.include_cfs, + "parameters": self.include_parameters, + } + + @property + def last_run_summary(self) -> Optional[dict]: + """Metadata from the last successful ``calculate`` call, or ``None``.""" + if self.last_run_includes is None: + return None + return { + "scenario": self.last_run_scenario, + "iterations": self.iterations, + "seed": self.seed, + "includes": self.last_run_includes, + } + @property def func_units_dict(self) -> dict: """Return a dictionary of reference flows (key, demand).""" diff --git a/activity_browser/bwutils/montecarlo_matrix_utils_patch.py b/activity_browser/bwutils/montecarlo/matrix_patch.py similarity index 95% rename from activity_browser/bwutils/montecarlo_matrix_utils_patch.py rename to activity_browser/bwutils/montecarlo/matrix_patch.py index 8e7e363d5..3a5c106ff 100644 --- a/activity_browser/bwutils/montecarlo_matrix_utils_patch.py +++ b/activity_browser/bwutils/montecarlo/matrix_patch.py @@ -10,6 +10,9 @@ Upstream fix: ``contrib/brightway-upstream/matrix_utils-resource_group.patch`` """ +#TODO remove this patch once bw2data 4.8 is released +# see also: https://github.com/brightway-lca/brightway2-data/pull/272 + from __future__ import annotations import numpy as np diff --git a/activity_browser/bwutils/montecarlo/scenarios.py b/activity_browser/bwutils/montecarlo/scenarios.py new file mode 100644 index 000000000..cde61ab10 --- /dev/null +++ b/activity_browser/bwutils/montecarlo/scenarios.py @@ -0,0 +1,49 @@ +"""Build scenario overlays from dataframes (fallback when ``SuperstructureMLCA`` is unavailable).""" + +# TODO: this should eventually be replaced by a "datapackage" based solution where bw datapackages are being used (For Montecarlo LCA incrementally for scenario data, uncertainty data and parameter uncertainty data) + +from __future__ import annotations + +from typing import Optional + +import bw2calc as bc +import pandas as pd + +from activity_browser.bwutils.superstructure.scenario_overlay import ( + ScenarioOverlay, + matrix_indices_for_multilca, + uncertainty_flags_for_indices, +) +from activity_browser.bwutils.superstructure.dataframe import ( + arrays_from_indexed_superstructure, + filter_databases_indexed_superstructure, + scenario_names_from_df, +) + + +def build_overlay_from_df( + lca: bc.MultiLCA, + scenario_df: pd.DataFrame, + databases: set[str], + scenario: Optional[str | int] = None, +) -> Optional[ScenarioOverlay]: + """Build overlay from a scenario dataframe (tests / fallback without ``SuperstructureMLCA``).""" + df = filter_databases_indexed_superstructure(scenario_df, databases) + names = scenario_names_from_df(df) + if not names or df.empty: + return None + + if scenario is None: + scenario = names[0] + elif isinstance(scenario, int): + scenario = names[scenario] + + indices, _ = arrays_from_indexed_superstructure(df) + amounts = df[scenario].to_numpy(dtype=float) + return ScenarioOverlay( + name=scenario, + indices=indices, + matrix_indices=matrix_indices_for_multilca(lca, indices), + amounts=amounts, + uncertain=uncertainty_flags_for_indices(indices), + ) diff --git a/activity_browser/bwutils/multilca.py b/activity_browser/bwutils/multilca.py index a65f544b6..87d102115 100644 --- a/activity_browser/bwutils/multilca.py +++ b/activity_browser/bwutils/multilca.py @@ -26,6 +26,19 @@ ca = ABContributionAnalysis() +def databases_for_fu_keys(fu_activity_keys: Iterable) -> set[str]: + """Dependent databases reachable from reference-flow ``(database, code)`` keys.""" + + def get_dependents(dbs: set, dependents: list) -> set: + for dep in (bd.databases[db].get("depends", []) for db in dependents): + if not dbs.issuperset(dep): + dbs = get_dependents(dbs.union(dep), dep) + return dbs + + dbs = set(f[0] for f in fu_activity_keys) + return get_dependents(dbs, list(dbs)) + + def _load_cs(obj, inv: list, ia: list) -> None: """Brightway 2.5 inv/ia keys and full label dicts on ``obj``.""" obj.func_units = list(inv) @@ -43,8 +56,10 @@ def _load_cs(obj, inv: list, ia: list) -> None: def setup_index(key) -> int | None: """Parse a contribution setup key (reference flow / method / scenario) to an int index.""" - if isinstance(key, int): - return key + if isinstance(key, bool): + return None + if isinstance(key, (int, np.integer)): + return int(key) if isinstance(key, str) and key.strip().isdigit(): return int(key.strip()) return None @@ -298,20 +313,7 @@ def func_units_dict(self) -> dict: @property def all_databases(self) -> set: """Get all databases linked to the reference flows.""" - - def get_dependents(dbs: set, dependents: list) -> set: - for dep in (bd.databases[db].get("depends", []) for db in dependents): - if not dbs.issuperset(dep): - dbs = get_dependents(dbs.union(dep), dep) - return dbs - - dbs = set(f[0] for f in self.fu_activity_keys) - dbs = get_dependents(dbs, list(dbs)) - # In rare cases, the default biosphere is not found as a dependency, see: - # https://github.com/LCA-ActivityBrowser/activity-browser/issues/298 - # Always include it. - # dbs.add(bd.config.biosphere) # commented out because biospheres aren't 'biosphere3' by default anymore - return dbs + return databases_for_fu_keys(self.fu_activity_keys) def get_results_for_method(self, index: int = 0) -> pd.DataFrame: data = self.lca_scores[:, index] diff --git a/activity_browser/bwutils/parameters/README.md b/activity_browser/bwutils/parameters/README.md index 2859ec297..c96dbe94c 100644 --- a/activity_browser/bwutils/parameters/README.md +++ b/activity_browser/bwutils/parameters/README.md @@ -26,6 +26,7 @@ LCI / LCIA → scores |--------|------| | `manager.py` | `ParameterManager`, `MonteCarloParameterManager` — formula evaluation order (project → database → activity → exchanges) | | `parameter_montecarlo.py` | Map recalculated amounts to `bw2calc` matrix indices; `functional_sqlite` process ↔ product via `bw_functional` | +| `formula_exchanges.py` | Rebuild the parameterized-flow index on database write (`INDEX_FLOW_CAP` = 1,000 outgoing flows); Parameterized Flows reads that index | | `utils.py` (parent `bwutils`) | `Parameter`, `Parameters`, `StaticParameters`, `Index`, `Indices` | `montecarlo.MonteCarloLCA` wires the hook and sets `keep_first_iteration_flag = False` (every iteration is sampled). @@ -52,6 +53,8 @@ Parameter **scenarios** (`convert_parameter_to_flow_scenarios.py`) use `activity - `commontasks.parameters_in_scope` — UI parameter scope - `superstructure/convert_parameter_to_flow_scenarios.py` — scenario conversion (separate from MC hook) +- `rebuild_parameterized_flow_index` — fill Brightway’s `ParameterizedExchange` index after a database write (ADR-0010) +- `indexed_parameterized_flows` — list index rows for the Parameterized Flows table ## Future work diff --git a/activity_browser/bwutils/parameters/formula_exchanges.py b/activity_browser/bwutils/parameters/formula_exchanges.py new file mode 100644 index 000000000..28f22bdae --- /dev/null +++ b/activity_browser/bwutils/parameters/formula_exchanges.py @@ -0,0 +1,146 @@ +"""Rebuild Brightway's parameterized-flow index after a database write.""" + +from __future__ import annotations + +import bw2data as bd +from bw2data.backends import ExchangeDataset +from bw2data.errors import UnknownObject +from bw2data.parameters import ActivityParameter, DatabaseParameter, ParameterizedExchange + +# Skip rebuild when outgoing flows exceed this and the database has no parameters. +INDEX_FLOW_CAP = 1000 + + +def flow_formula(exc: ExchangeDataset) -> str: + """Return the stripped formula on a flow document, or ``""``. + + Parameters + ---------- + exc : ExchangeDataset + Brightway flow document. + """ + data = getattr(exc, "data", None) + if isinstance(data, dict): + formula = str(data.get("formula") or "").strip() + if formula: + return formula + return str(getattr(exc, "formula", "") or "").strip() + + +def _process_types() -> frozenset[str]: + return frozenset(bd.labels.process_node_types) | {"multifunctional"} + + +def _should_rebuild(database: str) -> bool: + has_params = ( + DatabaseParameter.select().where(DatabaseParameter.database == database).count() + or ActivityParameter.select().where(ActivityParameter.database == database).count() + ) + if has_params: + return True + return ( + ExchangeDataset.select() + .where(ExchangeDataset.output_database == database) + .count() + <= INDEX_FLOW_CAP + ) + + +def _delete_index_rows_for_database(database: str) -> None: + groups = [ + row.group + for row in ActivityParameter.select(ActivityParameter.group).where( + ActivityParameter.database == database + ) + ] + if groups: + ParameterizedExchange.delete().where( + ParameterizedExchange.group << groups + ).execute() + outgoing_ids = [ + row.id + for row in ExchangeDataset.select(ExchangeDataset.id).where( + ExchangeDataset.output_database == database + ) + ] + if outgoing_ids: + ParameterizedExchange.delete().where( + ParameterizedExchange.exchange << outgoing_ids + ).execute() + + +def index_parameterized_flows_for_process(key: tuple) -> None: + """Index formula-bearing flows on one process and recalculate that group. + + Parameters + ---------- + key : tuple + Process ``(database, code)``. Non-process nodes are ignored. + """ + act = bd.get_activity(key) + if act.get("type", "process") not in _process_types(): + return + ap = ActivityParameter.get_or_none(database=key[0], code=key[1]) + group = ap.group if ap else act.id + with bd.parameters.db.atomic(): + bd.parameters.remove_exchanges_from_group(group, act) + bd.parameters.add_exchanges_to_group(group, act) + ActivityParameter.recalculate_exchanges(group) + + +def rebuild_parameterized_flow_index(database: str) -> None: + """Rebuild Brightway's parameterized-flow index for one database. + + Skips databases with more than ``INDEX_FLOW_CAP`` outgoing flows and no + database or activity parameters. Does not delete project, database, or + activity parameters. + + Parameters + ---------- + database : str + Name of the database that was written. + """ + if database not in bd.databases or not _should_rebuild(database): + return + + _delete_index_rows_for_database(database) + + process_codes = { + exc.output_code + for exc in ExchangeDataset.select().where( + ExchangeDataset.output_database == database + ) + if flow_formula(exc) + } + for code in process_codes: + try: + index_parameterized_flows_for_process((database, code)) + except UnknownObject: + continue + + +def indexed_parameterized_flows(): + """Yield parameterized-flow index rows as dicts. + + Each dict has ``formula``, ``amount``, ``comment``, ``uncertainty``, + ``input_key``, ``output_key``, and ``exchange`` (a live proxy, or ``None``). + """ + for pe in ParameterizedExchange.select(): + try: + doc = ExchangeDataset.get_by_id(int(pe.exchange)) + except Exception: + continue + data = doc.data if isinstance(doc.data, dict) else {} + try: + exchange = bd.Edge(document=doc) + except Exception: + exchange = None + yield { + "formula": pe.formula or flow_formula(doc), + "amount": data.get("amount"), + "comment": data.get("comment"), + "uncertainty": data.get("uncertainty") if isinstance(data.get("uncertainty"), dict) else {}, + "input_key": (doc.input_database, doc.input_code), + "output_key": (doc.output_database, doc.output_code), + "exchange": exchange, + } diff --git a/activity_browser/bwutils/parameters/parameter_montecarlo.py b/activity_browser/bwutils/parameters/parameter_montecarlo.py index 0406376b0..0d3ca050a 100644 --- a/activity_browser/bwutils/parameters/parameter_montecarlo.py +++ b/activity_browser/bwutils/parameters/parameter_montecarlo.py @@ -12,7 +12,7 @@ from __future__ import annotations -from typing import Any, List, Optional, Tuple +from typing import Any, Callable, List, Optional, Tuple import bw2calc as bc import bw2data as bd @@ -143,6 +143,50 @@ def product_row_in_lca( return _matrix_index(lca.dicts.product, database, code, activity_id) +def matrix_coords_for_exchange( + lca: bc.MultiLCA, + *, + flow_type: str, + input_database: str, + input_code: str, + output_database: str, + output_code: str, + input_id: Optional[int] = None, + output_id: Optional[int] = None, +) -> Optional[Tuple[str, int, int]]: + """Return ``(matrix_name, row, col)`` for a technosphere or biosphere exchange.""" + if input_id is None: + input_id = activity_id_from_key((input_database, input_code)) + if output_id is None: + output_id = activity_id_from_key((output_database, output_code)) + + if flow_type in bd.labels.biosphere_edge_types: + bio_row = lca.dicts.biosphere.get(input_id) + act_col = activity_col_in_lca(lca, output_database, output_code, output_id) + if bio_row is None or act_col is None: + return None + return ("biosphere_matrix", bio_row, act_col) + + prod_row = product_row_in_lca(lca, input_database, input_code, input_id) + if prod_row is None: + prod_row = product_row_in_lca(lca, output_database, output_code, output_id) + act_col = activity_col_in_lca(lca, output_database, output_code, output_id) + if prod_row is None or act_col is None: + return None + return ("technosphere_matrix", prod_row, act_col) + + +def write_matrix_amount( + lca: bc.MultiLCA, matrix_name: str, row: int, col: int, amount: float +) -> None: + if matrix_name == "technosphere_matrix": + lca.technosphere_matrix[row, col] = amount + if hasattr(lca, "solver"): + delattr(lca, "solver") + else: + lca.biosphere_matrix[row, col] = amount + + def exchange_from_param_row(row: np.void) -> ExchangeDataset: """Load the ``ExchangeDataset`` row described by a parameter MC numpy row.""" inp, out = row["input"], row["output"] @@ -170,14 +214,18 @@ def matrix_coords_for_param_row( def _biosphere_matrix_coords( lca: bc.MultiLCA, row: np.void ) -> Optional[Tuple[str, int, int]]: - input_id = activity_id_from_key(row["input"]) + in_db, in_code = activity_key_parts(row["input"]) out_db, out_code = activity_key_parts(row["output"]) - output_id = activity_id_from_key(row["output"]) - bio_row = lca.dicts.biosphere.get(input_id) - act_col = activity_col_in_lca(lca, out_db, out_code, output_id) - if bio_row is None or act_col is None: - return None - return ("biosphere_matrix", bio_row, act_col) + return matrix_coords_for_exchange( + lca, + flow_type="biosphere", + input_database=in_db, + input_code=in_code, + output_database=out_db, + output_code=out_code, + input_id=activity_id_from_key(row["input"]), + output_id=activity_id_from_key(row["output"]), + ) def _technosphere_matrix_coords( @@ -185,16 +233,16 @@ def _technosphere_matrix_coords( ) -> Optional[Tuple[str, int, int]]: in_db, in_code = activity_key_parts(row["input"]) out_db, out_code = activity_key_parts(row["output"]) - input_id = activity_id_from_key(row["input"]) - output_id = activity_id_from_key(row["output"]) - - prod_row = product_row_in_lca(lca, in_db, in_code, input_id) - if prod_row is None: - prod_row = product_row_in_lca(lca, out_db, out_code, output_id) - act_col = activity_col_in_lca(lca, out_db, out_code, output_id) - if prod_row is None or act_col is None: - return None - return ("technosphere_matrix", prod_row, act_col) + return matrix_coords_for_exchange( + lca, + flow_type="technosphere", + input_database=in_db, + input_code=in_code, + output_database=out_db, + output_code=out_code, + input_id=activity_id_from_key(row["input"]), + output_id=activity_id_from_key(row["output"]), + ) def signed_exchange_amount(row: np.void) -> float: @@ -213,28 +261,26 @@ def apply_parameter_exchanges(lca: bc.MultiLCA, param_rows: np.ndarray) -> int: if coords is None: continue matrix_name, i, j = coords - amount = signed_exchange_amount(row) - if matrix_name == "technosphere_matrix": - lca.technosphere_matrix[i, j] = amount - else: - lca.biosphere_matrix[i, j] = amount + write_matrix_amount(lca, matrix_name, i, j, signed_exchange_amount(row)) updated += 1 # if updated: # logger.debug("Parameter MC updated {} matrix cells".format(updated)) return updated -def bind_parameter_hook(lca: bc.MultiLCA, monte_carlo_lca: Any) -> None: - """ - Attach ``after_matrix_iteration`` on ``lca`` to apply parameter draws from ``monte_carlo_lca``. - - Expects ``monte_carlo_lca.include_parameters`` and ``monte_carlo_lca.parameter_mc_manager``. - """ +def bind_parameter_hook( + lca: bc.MultiLCA, + monte_carlo_lca: Any, + *, + before_parameters: Optional[Callable[[bc.MultiLCA], None]] = None, +) -> None: + """Attach ``after_matrix_iteration`` for optional pre-step and parameter draws.""" def after_matrix_iteration() -> None: + if before_parameters is not None: + before_parameters(lca) manager = monte_carlo_lca.parameter_mc_manager - if not monte_carlo_lca.include_parameters or manager is None: - return - apply_parameter_exchanges(lca, manager.next()) + if monte_carlo_lca.include_parameters and manager is not None: + apply_parameter_exchanges(lca, manager.next()) lca.after_matrix_iteration = after_matrix_iteration diff --git a/activity_browser/bwutils/pedigree.py b/activity_browser/bwutils/pedigree.py index 4b49ac8ce..fec977177 100644 --- a/activity_browser/bwutils/pedigree.py +++ b/activity_browser/bwutils/pedigree.py @@ -13,6 +13,7 @@ smoothly into the related uncertainty distributions. """ import math +from typing import NamedTuple from pprint import pformat from bw2data.parameters import ParameterBase @@ -83,3 +84,208 @@ def factors_as_tuple(self): def __repr__(self) -> str: return "Empty Pedigree Matrix" if not self.factors else pformat(self.factors) + + +SCORE_KEYS = ( + "reliability", + "completeness", + "temporal correlation", + "geographical correlation", + "further technological correlation", +) +BASIC_UNCERTAINTY_KEY = "basic uncertainty" + + +def infer_basic_uncertainty(recipe: dict, scale: float) -> float | None: + """Recover basic uncertainty from a lognormal scale and pedigree scores. + + Inverse of ``PedigreeMatrix.calculate``. Returns ``None`` when scores are + unusable or the scale is tighter than the scores alone allow. + """ + try: + scores = {key: recipe[key] for key in SCORE_KEYS if key in recipe} + matrix = PedigreeMatrix.from_dict(scores) + pedigree_ssq = sum(math.log(x) ** 2 for x in matrix.get_values()) + except (AssertionError, KeyError, TypeError, ValueError): + return None + residual = (2.0 * float(scale)) ** 2 - pedigree_ssq + if residual < 0 or math.isnan(residual): + return None + return math.exp(math.sqrt(residual)) + + +def _scores_only(recipe: dict | None) -> dict: + if not recipe: + return {} + return {key: int(recipe[key]) for key in SCORE_KEYS if key in recipe} + + +def recipe_is_usable(recipe: dict | None) -> bool: + if not recipe: + return False + scores = _scores_only(recipe) + if set(scores) != set(SCORE_KEYS): + return False + return all(1 <= scores[key] <= 5 for key in SCORE_KEYS) + + +def recipe_for_storage(recipe: dict, stored: dict | None = None) -> dict: + out = _scores_only(recipe) + basic = recipe.get(BASIC_UNCERTAINTY_KEY, 1.0) + out[BASIC_UNCERTAINTY_KEY] = 1.0 if basic is None else float(basic) + if stored and "sample size" in stored: + out["sample size"] = stored["sample size"] + return out + + +def pedigree_scores_suffix(recipe: dict | None) -> str: + """Cell text for stored scores (not basic uncertainty). Empty if unusable.""" + if not recipe_is_usable(recipe): + return "" + scores = _scores_only(recipe) + joined = ", ".join(str(scores[key]) for key in SCORE_KEYS) + return f"pedigree: {joined}" + + +def display_basic_uncertainty( + recipe: dict | None, *, scale: float, uncertainty_type: int +) -> float: + """Basic uncertainty to show in the dialog: stored, else inferred, else 1.""" + if recipe: + stored = recipe.get(BASIC_UNCERTAINTY_KEY) + if stored is not None: + try: + value = float(stored) + if math.isfinite(value) and value > 0: + return value + except (TypeError, ValueError): + pass + try: + from stats_arrays.distributions import LognormalUncertainty + + is_lognormal = int(uncertainty_type) == LognormalUncertainty.id + except Exception: + is_lognormal = int(uncertainty_type) == 2 + if is_lognormal: + inferred = infer_basic_uncertainty(recipe, scale) + if inferred is not None: + return inferred + return 1.0 + + +def _copy_sampled(sampled: dict) -> dict: + return {key: value for key, value in sampled.items() if key != "pedigree"} + + +def _default_editor_recipe() -> dict: + return {key: 1 for key in SCORE_KEYS} | {BASIC_UNCERTAINTY_KEY: 1.0} + + +def _editor_recipe_from_stored(stored: dict | None, sampled: dict) -> dict: + if not recipe_is_usable(stored): + return _default_editor_recipe() + try: + scale = float(sampled.get("scale", float("nan"))) + uncertainty_type = int(sampled.get("uncertainty type") or 0) + except (TypeError, ValueError): + scale, uncertainty_type = float("nan"), 0 + basic = display_basic_uncertainty( + stored, scale=scale, uncertainty_type=uncertainty_type + ) + return {**_scores_only(stored), BASIC_UNCERTAINTY_KEY: basic} + + +def _scale_from_recipe(recipe: dict) -> float: + matrix = PedigreeMatrix.from_dict(_scores_only(recipe)) + basic = recipe.get(BASIC_UNCERTAINTY_KEY, 1.0) + return matrix.calculate(1.0 if basic is None else float(basic)) + + +class PedigreeEditSession: + """In-dialog pedigree mode: check/uncheck/clear without Qt.""" + + def __init__(self, sampled: dict, stored: dict | None): + self.stored = stored if isinstance(stored, dict) else None + self.sampled = _copy_sampled(sampled) + self.use_pedigree = False + self.recipe_cleared = False + self.snapshot = None + self.recipe = _editor_recipe_from_stored(self.stored, self.sampled) + + def check_use(self) -> None: + if self.use_pedigree: + return + self.recipe_cleared = False + self.recipe = _editor_recipe_from_stored(self.stored, self.sampled) + self.snapshot = _copy_sampled(self.sampled) + self.use_pedigree = True + self._apply_recipe_to_sampled() + + def uncheck_use(self) -> None: + self._leave_use(restore=True, cleared=False) + + def stop_using_keep_sampled(self) -> None: + self._leave_use(restore=False, cleared=False) + + def clear(self) -> None: + self._leave_use(restore=True, cleared=True) + + def edit_recipe(self, recipe: dict) -> None: + if not self.use_pedigree: + return + self.recipe = dict(recipe) + self._apply_recipe_to_sampled() + + def set_sampled(self, sampled: dict) -> None: + self.sampled = _copy_sampled(sampled) + + def outcome(self) -> dict: + return { + "uncertainty": _copy_sampled(self.sampled), + "pedigree_applying": bool(self.use_pedigree), + "recipe_cleared": bool(self.recipe_cleared), + "recipe": None if self.recipe_cleared else dict(self.recipe), + } + + def _leave_use(self, *, restore: bool, cleared: bool) -> None: + if not self.use_pedigree: + return + if restore and self.snapshot is not None: + self.sampled = _copy_sampled(self.snapshot) + self.snapshot = None + self.use_pedigree = False + self.recipe_cleared = cleared + self.recipe = _editor_recipe_from_stored(self.stored, self.sampled) + + def _apply_recipe_to_sampled(self) -> None: + from stats_arrays.distributions import LognormalUncertainty + + self.sampled["uncertainty type"] = LognormalUncertainty.id + self.sampled["scale"] = _scale_from_recipe(self.recipe) + + +class PedigreeEditResult(NamedTuple): + write: dict + delete: tuple[str, ...] = () + + +def resolve_pedigree_edit(stored: dict | None, outcome: dict) -> PedigreeEditResult: + """Decide sampled-uncertainty writes and pedigree deletes from a dialog outcome.""" + write = { + key: value + for key, value in (outcome.get("uncertainty") or {}).items() + if key != "pedigree" + } + if outcome.get("recipe_cleared"): + return PedigreeEditResult(write=write, delete=("pedigree",)) + recipe = outcome.get("recipe") + if not outcome.get("pedigree_applying"): + return PedigreeEditResult(write=write) + if not recipe_is_usable(recipe): + raise ValueError("Cannot apply pedigree without a usable recipe") + from stats_arrays.distributions import LognormalUncertainty + + write["uncertainty type"] = LognormalUncertainty.id + write["scale"] = _scale_from_recipe(recipe) + write["pedigree"] = recipe_for_storage(recipe, stored) + return PedigreeEditResult(write=write) diff --git a/activity_browser/bwutils/superstructure/__init__.py b/activity_browser/bwutils/superstructure/__init__.py index bd6a4fbef..8162beea0 100644 --- a/activity_browser/bwutils/superstructure/__init__.py +++ b/activity_browser/bwutils/superstructure/__init__.py @@ -6,4 +6,12 @@ from .file_imports import ABCSVImporter, ABFeatherImporter, ABFileImporter from .manager import SuperstructureManager from .mlca import SuperstructureContributions, SuperstructureMLCA -from .utils import SUPERSTRUCTURE, _time_it_, edit_superstructure_for_string, parameters_to_sdf +from .utils import ( + SUPERSTRUCTURE, + _time_it_, + edit_superstructure_for_string, + is_flow_sdf_headers, + is_partial_flow_sdf_headers, + missing_superstructure_columns, + parameters_to_sdf, +) diff --git a/activity_browser/bwutils/superstructure/excel.py b/activity_browser/bwutils/superstructure/excel.py index b9e182668..9a581e57b 100644 --- a/activity_browser/bwutils/superstructure/excel.py +++ b/activity_browser/bwutils/superstructure/excel.py @@ -27,76 +27,75 @@ def get_sheet_names(document_path: Union[str, Path]) -> List[str]: logger.error("Given document uses an unknown encoding: {}".format(e)) -def get_header_index(document_path: Union[str, Path], import_sheet: int): - """Retrieves the line index for the column headers, will raise an - exception if not found in the first 10 rows. - """ - try: - with Path(document_path).open("rb") as f: - wb = openpyxl.load_workbook(filename=f, read_only=True) - sheet = wb.worksheets[import_sheet] - for i in range(10): - value = sheet.cell(i + 1, 1).value - # Skip SDF comment rows (first cell starts with '#'). - if isinstance(value, str) and not value.startswith("#"): - wb.close() - return i - except IndexError as e: - wb.close() - raise IndexError("Expected headers not found in file").with_traceback( - e.__traceback__ - ) - except UnicodeDecodeError as e: - logger.error("Given document uses an unknown encoding: {}".format(e)) - wb.close() - raise ValueError("Could not find required headers in given document sheet.") - - def valid_cols(name: str) -> bool: """True for data columns; names starting with '_' are SDF comment columns (not imported).""" return not str(name).startswith("_") +def _sdf_excel_skiprows(document_path: Union[str, Path], import_sheet: int) -> List[int]: + """Row indices to skip so ``header=0`` lands on the SDF header. + + Skips leading non-header rows and any row whose first cell starts with ``#``. + Header must appear within the first 10 rows (same rule as before). + """ + with Path(document_path).open("rb") as f: + wb = openpyxl.load_workbook(filename=f, read_only=True) + try: + sheet = wb.worksheets[import_sheet] + header_idx = None + hash_rows: List[int] = [] + for i, row in enumerate( + sheet.iter_rows(min_col=1, max_col=1, values_only=True) + ): + value = row[0] + if isinstance(value, str) and value.startswith("#"): + hash_rows.append(i) + continue + if header_idx is None and isinstance(value, str): + header_idx = i + if header_idx is None and i >= 9: + break + if header_idx is None: + raise ValueError( + "Could not find required headers in given document sheet." + ) + return sorted(set(range(header_idx)) | set(hash_rows)) + finally: + wb.close() + + def import_from_excel( document_path: Union[str, Path], import_sheet: int = 1 ) -> pd.DataFrame: - """Import all of the exchanges and their scenario amounts from a given - document and sheet index. - - The default index chosen represents the second sheet (first after the - 'information' sheet). + """Import scenario exchanges from an Excel sheet. - Comment rows: a '#' at the start of a row (pandas ``comment='#'``). - Comment columns: a column name starting with '_' (``usecols=valid_cols``). + Comment rows (first cell starts with ``#``) and comment columns (name + starts with ``_``) are excluded via ``skiprows`` / ``usecols`` — not + pandas ``comment='#'``, which breaks Excel headers under openpyxl. """ - data = pd.DataFrame({}) try: - header_idx = get_header_index(document_path, import_sheet) + skiprows = _sdf_excel_skiprows(document_path, import_sheet) with Path(document_path).open("rb") as f: data = pd.read_excel( f, sheet_name=import_sheet, - header=header_idx, + header=0, + skiprows=skiprows or None, usecols=valid_cols, - comment="#", na_values="", keep_default_na=False, engine="openpyxl", ) diff = SUPERSTRUCTURE.difference(data.columns) if not diff.empty: - raise ValueError( - "Missing required column(s) for superstructure: {}".format( - diff.to_list() - ) - ) + # Return the frame as-read so callers can report missing headers. + # Do not run key converters that require a complete SUPERSTRUCTURE. + return ensure_string_scenario_names(data) - # Convert specific columns that may have tuples as strings columns = ["from categories", "from key", "to categories", "to key"] data.loc[:, columns] = data[columns].map(convert_tuple_str) - # Scenario headers typed as numbers in Excel (e.g. 2025) must be strings. data = ensure_string_scenario_names(data) + return data except Exception as e: - # Caller (UI) decides how to surface failures; empty frame means "not this sheet". logger.debug("Excel scenario import failed for sheet {}: {}", import_sheet, e) - return data + return pd.DataFrame({}) diff --git a/activity_browser/bwutils/superstructure/mlca.py b/activity_browser/bwutils/superstructure/mlca.py index 6e849a514..2a2cd37a7 100644 --- a/activity_browser/bwutils/superstructure/mlca.py +++ b/activity_browser/bwutils/superstructure/mlca.py @@ -17,6 +17,7 @@ filter_databases_indexed_superstructure, scenario_names_from_df) from .file_dialogs import ABPopup +from .scenario_overlay import ScenarioOverlay, uncertainty_flags_for_indices try: from bw2calc.matrices import TechnosphereBiosphereMatrixBuilder as MB @@ -34,6 +35,7 @@ class SuperstructureMLCA(MLCA): "biosphere": "biosphere_matrix", "technosphere": "technosphere_matrix", "production": "technosphere_matrix", + "substitution": "technosphere_matrix", } def __init__(self, cs_name: str, df: pd.DataFrame, cs: dict | None = None): @@ -56,6 +58,7 @@ def __init__(self, cs_name: str, df: pd.DataFrame, cs: dict | None = None): self.defaults = { "technosphere": "default_technosphere_matrix", "production": "default_technosphere_matrix", + "substitution": "default_technosphere_matrix", "biosphere": "default_biosphere_matrix", } @@ -81,6 +84,7 @@ def __init__(self, cs_name: str, df: pd.DataFrame, cs: dict | None = None): ], ) self.indices_to_matrix() + self.exchange_uncertain = uncertainty_flags_for_indices(self.indices) # Construct an index dictionary similar to fu_index and method_index self._current_index = 0 @@ -107,6 +111,19 @@ def __init__(self, cs_name: str, df: pd.DataFrame, cs: dict | None = None): ) ) + def scenario_overlay(self, scenario: str | int) -> ScenarioOverlay: + """Precomputed matrix overlay for one scenario (reused by Monte Carlo).""" + if isinstance(scenario, int): + scenario = self.scenario_names[scenario] + col = self.scenario_index[scenario] + return ScenarioOverlay( + name=scenario, + indices=self.indices, + matrix_indices=self.matrix_indices, + amounts=self.values[:, col].copy(), + uncertain=self.exchange_uncertain, + ) + @property def current(self) -> int: return self._current_index diff --git a/activity_browser/bwutils/superstructure/scenario_overlay.py b/activity_browser/bwutils/superstructure/scenario_overlay.py new file mode 100644 index 000000000..909d62030 --- /dev/null +++ b/activity_browser/bwutils/superstructure/scenario_overlay.py @@ -0,0 +1,155 @@ +"""Precomputed scenario matrix overlay (shared by SuperstructureMLCA and Monte Carlo).""" + +# TODO: this should eventually be replaced by a "datapackage" based solution where bw datapackages are being used (For Montecarlo LCA incrementally for scenario data, uncertainty data and parameter uncertainty data) + +from __future__ import annotations + +from dataclasses import dataclass + +import bw2calc as bc +import bw2data as bd +import numpy as np +from bw2data.backends import ExchangeDataset +from stats_arrays import distributions as sa + +from activity_browser.bwutils.uncertainty import uncertainty_type_id +from activity_browser.bwutils.utils import Index +from activity_browser.bwutils.parameters.parameter_montecarlo import ( + matrix_coords_for_exchange, +) + +_DETERMINISTIC_UNCERTAINTY_TYPES = frozenset( + { + sa.UndefinedUncertainty.id, + sa.NoUncertainty.id, + } +) + +_MATRIX_NAMES = { + "biosphere": "biosphere_matrix", + "technosphere": "technosphere_matrix", + "production": "technosphere_matrix", + "substitution": "technosphere_matrix", +} + + +@dataclass(frozen=True) +class ScenarioOverlay: + """Precomputed scenario amounts and matrix coordinates for MC overlay.""" + + name: str + indices: np.ndarray + matrix_indices: np.ndarray + amounts: np.ndarray + uncertain: np.ndarray + + +def uncertainty_flags_for_indices(indices: np.ndarray) -> np.ndarray: + """Return a bool array: True where the exchange has MC-relevant uncertainty. + + Any ``stats_arrays`` distribution (type id >= 2) is resampled when the + technosphere/biosphere MC layer is on; only undefined/no-uncertainty are + treated as deterministic for scenario pinning. + """ + flags = np.zeros(len(indices), dtype=bool) + for i, index in enumerate(indices): + flags[i] = ( + exchange_uncertainty_type(index) not in _DETERMINISTIC_UNCERTAINTY_TYPES + ) + return flags + + +def matrix_indices_for_multilca( + lca: bc.MultiLCA, indices: np.ndarray +) -> np.ndarray: + """Resolve ``(row, col, flip)`` on a ``MultiLCA`` (once per calculate).""" + result = np.zeros( + len(indices), + dtype=[("row", np.uint32), ("col", np.uint32), ("flip", np.bool_)], + ) + for i, index in enumerate(indices): + coords = matrix_coords_for_exchange( + lca, + flow_type=index.flow_type, + input_database=index.input.database, + input_code=index.input.code, + output_database=index.output.database, + output_code=index.output.code, + input_id=index.input_id, + output_id=index.output_id, + ) + if coords is None: + continue + _, row, col = coords + result[i] = (row, col, index.flip) + return result + + +def apply_scenario_overlay( + lca: bc.MultiLCA, + overlay: ScenarioOverlay, + *, + include_technosphere: bool, + include_biosphere: bool, + repin_only: bool = False, +) -> None: + """Write scenario amounts into LCA matrices. + + Initial pass (``repin_only=False``): pin every scenario cell that should + hold a fixed amount (layer off, or layer on but exchange deterministic). + + Re-pin pass (``repin_only=True``): only layers being MC-resampled; used after + ``next()`` restored DB draws on uncertain cells while leaving pinned cells to + be overwritten from the datapackage. + """ + flow_types = np.array([idx.flow_type for idx in overlay.indices]) + for kind in np.unique(flow_types): + mask = flow_types == kind + idx = overlay.matrix_indices[mask] + sample = overlay.amounts[mask].copy() + uncertain = overlay.uncertain[mask] + + valid = ~np.isnan(sample) + idx = idx[valid] + sample = sample[valid] + uncertain = uncertain[valid] + if sample.size == 0: + continue + + is_bio = kind in bd.labels.biosphere_edge_types + layer_on = include_biosphere if is_bio else include_technosphere + if repin_only: + if not layer_on: + continue + keep = ~uncertain + idx = idx[keep] + sample = sample[keep] + elif layer_on: + keep = ~uncertain + idx = idx[keep] + sample = sample[keep] + if sample.size == 0: + continue + + flip = idx["flip"] + sample[flip] *= -1 + + matrix_name = _MATRIX_NAMES.get(kind, "technosphere_matrix") + matrix = getattr(lca, matrix_name) + matrix[idx["row"], idx["col"]] = sample + + if matrix_name == "technosphere_matrix" and hasattr(lca, "solver"): + delattr(lca, "solver") + + +def exchange_uncertainty_type(index: Index) -> int: + try: + exc = ExchangeDataset.get( + ExchangeDataset.input_code == index.input.code, + ExchangeDataset.input_database == index.input.database, + ExchangeDataset.output_code == index.output.code, + ExchangeDataset.output_database == index.output.database, + ) + return uncertainty_type_id(exc.data or {}) + except ExchangeDataset.DoesNotExist: + return 0 diff --git a/activity_browser/bwutils/superstructure/utils.py b/activity_browser/bwutils/superstructure/utils.py index 7ec2941a4..7eef44a40 100644 --- a/activity_browser/bwutils/superstructure/utils.py +++ b/activity_browser/bwutils/superstructure/utils.py @@ -33,6 +33,23 @@ SCENARIO_NAME_JOIN = " | " +def missing_superstructure_columns(columns) -> list[str]: + """Return required SDF header names absent from ``columns`` (stable order).""" + return SUPERSTRUCTURE.difference(pd.Index(columns)).tolist() + + +def is_flow_sdf_headers(columns) -> bool: + """True when all SUPERSTRUCTURE columns are present (extra scenario cols OK).""" + return not missing_superstructure_columns(columns) + + +def is_partial_flow_sdf_headers(columns, *, min_matches: int = 8) -> bool: + """True when headers look like an SDF attempt but are incomplete/misspelled.""" + cols = pd.Index(columns) + n = len(cols.intersection(SUPERSTRUCTURE)) + return n >= min_matches and not is_flow_sdf_headers(cols) + + def edit_superstructure_for_string( superstructure=SUPERSTRUCTURE, sep="
", fhighlight="" ): diff --git a/activity_browser/bwutils/uncertainty.py b/activity_browser/bwutils/uncertainty.py index 794891a72..fd9e5876f 100644 --- a/activity_browser/bwutils/uncertainty.py +++ b/activity_browser/bwutils/uncertainty.py @@ -26,6 +26,8 @@ from bw2data.proxies import ExchangeProxyBase from stats_arrays import UncertaintyBase, UndefinedUncertainty, uncertainty_choices as uc +from activity_browser.bwutils.pedigree import pedigree_scores_suffix + # Cleared uncertainty state for remove-uncertainty actions and dialog defaults. EMPTY_UNCERTAINTY = { "uncertainty type": UndefinedUncertainty.id, @@ -287,13 +289,33 @@ def uncertainty_parameters_summary(source) -> str: return "; ".join(parts) -def uncertainty_cell_summary(source) -> str: +def uncertainty_cell_summary(source, pedigree=None) -> str: """Single-table-cell text: distribution type and parameters (``Type; param: val; …``).""" type_name = uncertainty_type_name(source) params = uncertainty_parameters_summary(source) if type_name and params: - return f"{type_name}; {params}" - return type_name or params + text = f"{type_name}; {params}" + else: + text = type_name or params + recipe = pedigree + if recipe is None and isinstance(source, dict): + recipe = source.get("pedigree") + suffix = pedigree_scores_suffix(recipe) + if suffix and text: + return f"{text}; {suffix}" + return suffix or text + + +def uncertainty_initial_from_flow(exchange) -> dict: + """Sampled uncertainty plus stored pedigree recipe for the flow editor.""" + data = dict(getattr(exchange, "uncertainty", None) or {}) + try: + pedigree = exchange.get("pedigree") + except Exception: + pedigree = None + if pedigree: + data["pedigree"] = pedigree + return data class BaseUncertaintyInterface(abc.ABC): diff --git a/activity_browser/templates/README.md b/activity_browser/templates/README.md index c74352487..d97a62b1d 100644 --- a/activity_browser/templates/README.md +++ b/activity_browser/templates/README.md @@ -28,7 +28,7 @@ CSV files: header row, then notes on lines starting with `#` (ignored on import) Scenario import comments (Excel and CSV): -- **Rows:** start with `#` (ignored via pandas `comment="#"`). +- **Rows:** start with `#` in the first cell (CSV: `comment="#"`; Excel: `skiprows` — not `comment="#"`). - **Columns:** name starts with `_` (e.g. `_notes`; dropped via `usecols`). **Get template → flow-scenarios** always copies the empty starter file (does not generate from project parameters). diff --git a/activity_browser/ui/core/application.py b/activity_browser/ui/core/application.py index b376a8922..ed9d35566 100644 --- a/activity_browser/ui/core/application.py +++ b/activity_browser/ui/core/application.py @@ -1,4 +1,5 @@ import os +import sys from pathlib import Path from loguru import logger @@ -10,6 +11,24 @@ _OFFSCREEN_FLAGS = ("--disable-gpu", "--disable-gpu-compositing", "--no-sandbox") +# Qt WebEngine on Wayland can fail with blank Graph/Tree/Sankey panes +# ("Backend texture is not a Vulkan texture" / "Compositor returned null texture"). +# Confirmed workaround: disable Chromium GPU compositing while staying on Wayland. +_WAYLAND_FLAGS = ("--disable-gpu", "--disable-gpu-compositing") + + +def _is_linux_wayland() -> bool: + """True when running on Linux under a Wayland session (not forced xcb/offscreen).""" + if not sys.platform.startswith("linux"): + return False + qpa = os.environ.get("QT_QPA_PLATFORM", "").split(":")[0].lower() + if qpa in ("xcb", "offscreen", "minimal", "vnc"): + return False + if qpa.startswith("wayland"): + return True + if os.environ.get("XDG_SESSION_TYPE", "").lower() == "wayland": + return True + return bool(os.environ.get("WAYLAND_DISPLAY")) def _webengine_flags(*add: str, drop: tuple[str, ...] = ()) -> None: @@ -17,6 +36,7 @@ def _webengine_flags(*add: str, drop: tuple[str, ...] = ()) -> None: skip = set(drop) order = ( *(_OFFSCREEN_FLAGS if os.environ.get("QT_QPA_PLATFORM") == "offscreen" else ()), + *(_WAYLAND_FLAGS if _is_linux_wayland() else ()), *os.environ.get("QTWEBENGINE_CHROMIUM_FLAGS", "").split(), *add, ) @@ -69,6 +89,11 @@ def set_icon(self): def pyside6_setup(self): from qtpy.QtWebEngineQuick import QtWebEngineQuick + if _is_linux_wayland(): + logger.info( + "Linux Wayland detected: disabling Qt WebEngine GPU compositing " + "so Graph / Tree / Sankey can render" + ) _webengine_flags() QtWebEngineQuick.initialize() diff --git a/activity_browser/ui/core/tree_model.py b/activity_browser/ui/core/tree_model.py index dbe18f3fe..a21b57323 100644 --- a/activity_browser/ui/core/tree_model.py +++ b/activity_browser/ui/core/tree_model.py @@ -297,6 +297,10 @@ def uncertainty_editor_read_only(self, index: QModelIndex) -> bool: """If True, :class:`~activity_browser.ui.dialogs.UncertaintyDialog` opens read-only.""" return False + def uncertainty_editor_enable_pedigree(self, index: QModelIndex) -> bool: + """If True, the uncertainty dialog shows the pedigree recipe (flows only).""" + return False + def isBranchNode(self, index: QModelIndex) -> bool: """Check if the given index represents a branch node (non-leaf).""" if not index.isValid(): diff --git a/activity_browser/ui/delegates/new_formula.py b/activity_browser/ui/delegates/new_formula.py index b118624bb..4ed661034 100644 --- a/activity_browser/ui/delegates/new_formula.py +++ b/activity_browser/ui/delegates/new_formula.py @@ -17,47 +17,53 @@ def sizeHint(self, option, index): def displayText(self, value, locale): return f"{value}" + def _scope_for_index(self, index): + if hasattr(index.internalPointer(), "scoped_parameters"): + return index.internalPointer().scoped_parameters + if hasattr(index.model(), "scoped_parameters"): + return index.model().scoped_parameters(index) + return {} + def paint(self, painter, option: QtWidgets.QStyleOptionViewItem, index): if index.data() is None: return super().paint(painter, option, index) - painter.save() - - if option.state & QtWidgets.QStyle.State_Selected: - painter.fillRect(option.rect, option.palette.color(option.palette.ColorRole.Highlight)) - painter.setPen(option.palette.color(option.palette.ColorRole.HighlightedText)) - else: - painter.setPen(Qt.NoPen) - - if hasattr(index.internalPointer(), 'scoped_parameters'): - scope = index.internalPointer().scoped_parameters - elif hasattr(index.model(), 'scoped_parameters'): - scope = index.model().scoped_parameters(index) - else: - scope = {} - - from activity_browser.ui.widgets import ABFormulaEdit - viewport = self.parent().findChild(QtWidgets.QWidget, "qt_scrollarea_viewport") - formula = ABFormulaEdit(viewport, scope, index.data(), simple=True) - - painter.setClipRect(option.rect) - painter.translate(option.rect.topLeft()) - - formula.setGeometry(option.rect) - formula.paint_text(painter) + try: + scope = self._scope_for_index(index) + except Exception: + logger.opt(exception=True).debug("Formula scope lookup failed during paint") + return super().paint(painter, option, index) - painter.restore() + painter.save() + try: + if option.state & QtWidgets.QStyle.State_Selected: + painter.fillRect(option.rect, option.palette.color(option.palette.ColorRole.Highlight)) + painter.setPen(option.palette.color(option.palette.ColorRole.HighlightedText)) + else: + painter.setPen(Qt.NoPen) + + from activity_browser.ui.widgets import ABFormulaEdit + viewport = self.parent().findChild(QtWidgets.QWidget, "qt_scrollarea_viewport") + formula = ABFormulaEdit(viewport, scope, index.data(), simple=True) + + painter.setClipRect(option.rect) + painter.translate(option.rect.topLeft()) + + formula.setGeometry(option.rect) + formula.paint_text(painter) + except Exception: + logger.opt(exception=True).debug("Formula delegate paint failed") + finally: + painter.restore() def createEditor(self, parent, option, index): from activity_browser.ui.widgets import ABFormulaEdit - if hasattr(index.internalPointer(), 'scoped_parameters'): - scope = index.internalPointer().scoped_parameters - elif hasattr(index.model(), 'scoped_parameters'): - scope = index.model().scoped_parameters(index) - else: + try: + scope = self._scope_for_index(index) + except Exception: + logger.opt(exception=True).debug("Formula scope lookup failed during edit") scope = {} - editor = ABFormulaEdit(parent, scope) - return editor + return ABFormulaEdit(parent, scope) def setEditorData(self, editor, index: QtCore.QModelIndex): """Populate the editor with data if editing an existing field.""" diff --git a/activity_browser/ui/delegates/uncertainty.py b/activity_browser/ui/delegates/uncertainty.py index 10fa1629c..0283342be 100644 --- a/activity_browser/ui/delegates/uncertainty.py +++ b/activity_browser/ui/delegates/uncertainty.py @@ -34,8 +34,15 @@ def createEditor(self, parent, option, index): ro_getter = getattr(model, "uncertainty_editor_read_only", None) if callable(ro_getter): read_only = bool(ro_getter(index)) + enable_pedigree = False + pe_getter = getattr(model, "uncertainty_editor_enable_pedigree", None) + if callable(pe_getter): + enable_pedigree = bool(pe_getter(index)) return UncertaintyDialog( - parent=app.main_window, initial=initial, read_only=read_only + parent=app.main_window, + initial=initial, + read_only=read_only, + enable_pedigree=enable_pedigree, ) def setEditorData(self, editor, index: QtCore.QModelIndex): diff --git a/activity_browser/ui/dialogs/uncertainty_dialog.py b/activity_browser/ui/dialogs/uncertainty_dialog.py index 280a7e527..0d51b0074 100644 --- a/activity_browser/ui/dialogs/uncertainty_dialog.py +++ b/activity_browser/ui/dialogs/uncertainty_dialog.py @@ -8,6 +8,13 @@ from qtpy import QtCore, QtGui, QtWidgets import stats_arrays as sa +from activity_browser.bwutils.pedigree import ( + BASIC_UNCERTAINTY_KEY, + PedigreeEditSession, + SCORE_KEYS, + recipe_is_usable, + resolve_pedigree_edit, +) from activity_browser.bwutils.uncertainty import ( DISTRIBUTIONS_WITH_CALCULATED_MEAN, EMPTY_UNCERTAINTY, @@ -31,6 +38,74 @@ "Complete them to preview the distribution and enable OK." ) +# Preview floor; the dialog opens taller so the plot can grow on first show. +_PLOT_MIN_HEIGHT = 180 +_DIALOG_OPEN_WIDTH = 540 +_DIALOG_OPEN_HEIGHT = 640 + + +def _field_text(val) -> str: + if val is None or (isinstance(val, float) and np.isnan(val)): + return "" + return str(val) + + +def _distribution_index(data: dict) -> int: + try: + uc_type = int(data.get("uncertainty type") or 0) + except (TypeError, ValueError): + uc_type = 0 + if uc_type < 0 or uc_type >= len(sa.uncertainty_choices): + return 0 + return uc_type + + +def _basic_uncertainty_value(value=1.0) -> float: + try: + basic = float(value) + except (TypeError, ValueError): + return 1.0 + return basic if np.isfinite(basic) and basic > 0 else 1.0 + + +_PEDIGREE_CHOICES = { + "reliability": [ + "1) Verified data based on measurements", + "2) Verified data partly based on assumptions", + "3) Non-verified data partly based on qualified measurements", + "4) Qualified estimate", + "5) Non-qualified estimate", + ], + "completeness": [ + "1) Representative relevant data from all sites, over an adequate period", + "2) Representative relevant data from >50% sites, over an adequate period", + "3) Representative relevant data from <50% sites OR >50%, but over shorter period", + "4) Representative relevant data from one site OR some sites but over shorter period", + "5) Representativeness unknown", + ], + "temporal correlation": [ + "1) Data less than 3 years old", + "2) Data less than 6 years old", + "3) Data less than 10 years old", + "4) Data less than 15 years old", + "5) Data age unknown or more than 15 years old", + ], + "geographical correlation": [ + "1) Data from area under study", + "2) Average data from larger area in which area under study is included", + "3) Data from area with similar production conditions", + "4) Data from area with slightly similar production conditions", + "5) Data from unknown OR distinctly different area", + ], + "further technological correlation": [ + "1) Data from enterprises, processes and materials under study", + "2) Data from processes and materials under study, different enterprise", + "3) Data from processes and materials under study from different technology", + "4) Data on related processes and materials", + "5) Data on related processes on lab scale OR from different technology", + ], +} + class UncertaintyDialog(QtWidgets.QDialog): """Single-step dialog for defining a stats_arrays uncertainty. @@ -44,17 +119,24 @@ class UncertaintyDialog(QtWidgets.QDialog): # array is a numpy structured array compatible with stats_arrays """ - def __init__(self, parent=None, initial: Optional[dict] = None, *, read_only: bool = False): + def __init__( + self, + parent=None, + initial: Optional[dict] = None, + *, + read_only: bool = False, + enable_pedigree: bool = False, + ): super().__init__(parent) self._read_only = read_only self.setWindowTitle("Set Uncertainty") self.setAttribute(QtCore.Qt.WA_DeleteOnClose) - - # State self.dist = None - self.result_array = None # Filled on accept - self.result_dict = None # Filled on accept + self.result_array = None + self.result_dict = None self.previous_dist_id: Optional[int] = None + self._updating_from_pedigree = False + self._session = None # Top: distribution selection box1 = QtWidgets.QGroupBox("Select the uncertainty distribution") @@ -68,6 +150,12 @@ def __init__(self, parent=None, initial: Optional[dict] = None, *, read_only: bo header_layout.setVerticalSpacing(4) header_layout.addWidget(QtWidgets.QLabel("Distribution:"), 0, 0) header_layout.addWidget(self.distribution, 0, 1) + self.use_pedigree = None + if enable_pedigree: + self.use_pedigree = QtWidgets.QCheckBox("Use pedigree") + self.use_pedigree.setChecked(False) + self.use_pedigree.toggled.connect(self._on_use_pedigree_toggled) + header_layout.addWidget(self.use_pedigree, 1, 0, 1, 2) box1.setLayout(header_layout) # Middle: parameters @@ -103,6 +191,7 @@ def __init__(self, parent=None, initial: Optional[dict] = None, *, read_only: bo self.scale_label = QtWidgets.QLabel("Sigma/scale:") self.scale = QtWidgets.QLineEdit() self.scale.setValidator(self.validator) + self.scale.textEdited.connect(self._on_scale_edited) self.scale.textEdited.connect(self._schedule_plot_refresh) self.shape_label = QtWidgets.QLabel("Shape:") @@ -162,6 +251,10 @@ def __init__(self, parent=None, initial: Optional[dict] = None, *, read_only: bo params_layout.addWidget(self.neg_samples_cb, row, 0, 1, 2) self.fields_box.setLayout(params_layout) + self.pedigree_box = self._build_pedigree_box() if enable_pedigree else None + if self.pedigree_box is not None: + self.pedigree_box.hide() + # Bottom: plot + status when preview is unavailable self.plot = SimpleDistributionPlot(self) self._plot_message = QtWidgets.QLabel() @@ -182,17 +275,18 @@ def __init__(self, parent=None, initial: Optional[dict] = None, *, read_only: bo self.buttons.accepted.connect(self._on_accept) self.buttons.rejected.connect(self.reject) - # Layout layout = QtWidgets.QVBoxLayout() layout.setSpacing(6) layout.setContentsMargins(12, 10, 12, 10) layout.addWidget(box1) layout.addWidget(self.fields_box) - # Stretch so the preview absorbs all extra height when the dialog is resized. + if self.pedigree_box is not None: + layout.addWidget(self.pedigree_box) layout.addWidget(self.plot, 1) layout.addWidget(self._plot_message) layout.addWidget(self.buttons) self.setLayout(layout) + self.setSizeGripEnabled(True) self._plot_refresh_timer = QtCore.QTimer(self) self._plot_refresh_timer.setSingleShot(True) @@ -201,6 +295,8 @@ def __init__(self, parent=None, initial: Optional[dict] = None, *, read_only: bo # Initialize values (defaults or provided initial) self._apply_initial(initial or {}) + if enable_pedigree: + self._load_pedigree(initial or {}) self._on_distribution_changed(self.distribution.currentIndex()) self._sync_mean_from_loc() self._generate_plot() @@ -210,58 +306,183 @@ def __init__(self, parent=None, initial: Optional[dict] = None, *, read_only: bo # ---------- Public API ---------- @staticmethod def get_uncertainty_array( - parent=None, initial: Optional[dict] = None, *, read_only: bool = False + parent=None, initial: Optional[dict] = None, *, read_only: bool = False, + enable_pedigree: bool = False, ) -> Tuple[bool, Optional[np.ndarray]]: - dlg = UncertaintyDialog(parent, initial=initial, read_only=read_only) + dlg = UncertaintyDialog( + parent, initial=initial, read_only=read_only, enable_pedigree=enable_pedigree + ) ok = dlg.exec_() == QtWidgets.QDialog.Accepted return ok, dlg.result_array if ok else None @staticmethod def get_uncertainty_dict( - parent=None, initial: Optional[dict] = None, *, read_only: bool = False + parent=None, initial: Optional[dict] = None, *, read_only: bool = False, + enable_pedigree: bool = False, ) -> Tuple[bool, Optional[dict]]: - dlg = UncertaintyDialog(parent, initial=initial, read_only=read_only) + dlg = UncertaintyDialog( + parent, initial=initial, read_only=read_only, enable_pedigree=enable_pedigree + ) ok = dlg.exec_() == QtWidgets.QDialog.Accepted return ok, dlg.result_dict if ok else None # ---------- Internal helpers ---------- def _apply_initial(self, initial: dict) -> None: - # Use EMPTY_UNCERTAINTY defaults, overridden by initial - data = {k: v for k, v in EMPTY_UNCERTAINTY.items()} - data.update(initial or {}) - # Do not load numerics that cannot be sampled (e.g. Student's T with df <= 0). + data = {**EMPTY_UNCERTAINTY, **(initial or {})} if not uncertainty_dict_is_sampleable(data): + data = { + **EMPTY_UNCERTAINTY, + "uncertainty type": _distribution_index(data), + } + self._write_sampled_fields(data) + + def _build_pedigree_box(self) -> QtWidgets.QGroupBox: + box = QtWidgets.QGroupBox("Pedigree") + self.clear_pedigree = QtWidgets.QPushButton("Clear pedigree") + self.clear_pedigree.clicked.connect(self._on_clear_pedigree) + self.pedigree_combos = {} + grid = QtWidgets.QGridLayout() + grid.setContentsMargins(8, 4, 8, 4) + grid.setVerticalSpacing(4) + for row, key in enumerate(SCORE_KEYS): + combo = QtWidgets.QComboBox() + combo.addItems(_PEDIGREE_CHOICES[key]) + combo.currentIndexChanged.connect(self._on_pedigree_recipe_edited) + self.pedigree_combos[key] = combo + grid.addWidget(QtWidgets.QLabel(key.capitalize()), row, 0) + grid.addWidget(combo, row, 1, 1, 2) + row = len(SCORE_KEYS) + self.basic_uncertainty = QtWidgets.QLineEdit("1") + self.basic_uncertainty.setValidator(self.validator) + self.basic_uncertainty.textEdited.connect(self._on_pedigree_recipe_edited) + grid.addWidget(QtWidgets.QLabel("Basic uncertainty"), row, 0) + grid.addWidget(self.basic_uncertainty, row, 1, 1, 2) + grid.addWidget(self.clear_pedigree, row + 1, 2) + box.setLayout(grid) + return box + + def _load_pedigree(self, initial: dict) -> None: + stored = initial.get("pedigree") + self._session = PedigreeEditSession( + initial, stored if isinstance(stored, dict) else None + ) + self._apply_recipe_widgets(self._session.recipe) + + def _apply_recipe_widgets(self, recipe: dict | None) -> None: + self._updating_from_pedigree = True + scores = recipe or {} + for key, combo in self.pedigree_combos.items(): try: - uc_type = int(data.get("uncertainty type", 0)) - except Exception: - uc_type = 0 - if uc_type < 0 or uc_type >= len(sa.uncertainty_choices): - uc_type = 0 - data = {k: v for k, v in EMPTY_UNCERTAINTY.items()} - data["uncertainty type"] = uc_type - # Distribution - try: - uc_type = int(data.get("uncertainty type", 0)) - except Exception: - uc_type = 0 - self.distribution.setCurrentIndex(uc_type) - # Fields (string form for QLineEdit) - def to_str(val): - if val is None or (isinstance(val, float) and np.isnan(val)): - return "" - return str(val) - - self.loc.setText(to_str(data.get("loc", np.nan))) - self.scale.setText(to_str(data.get("scale", np.nan))) - self.shape.setText(to_str(data.get("shape", np.nan))) - self.minimum.setText(to_str(data.get("minimum", np.nan))) - self.maximum.setText(to_str(data.get("maximum", np.nan))) + score = int(scores.get(key, 1)) + except (TypeError, ValueError): + score = 1 + combo.setCurrentIndex(min(max(score, 1), 5) - 1) + self.basic_uncertainty.setText( + str(_basic_uncertainty_value(scores.get(BASIC_UNCERTAINTY_KEY, 1.0))) + ) + self._updating_from_pedigree = False + + def _current_recipe(self) -> dict: + recipe = { + key: combo.currentIndex() + 1 + for key, combo in self.pedigree_combos.items() + } + recipe[BASIC_UNCERTAINTY_KEY] = _basic_uncertainty_value( + self.basic_uncertainty.text() + ) + return recipe + + def _write_sampled_fields(self, data: dict) -> None: + self.distribution.setCurrentIndex(_distribution_index(data)) + self.loc.setText(_field_text(data.get("loc", np.nan))) + self.scale.setText(_field_text(data.get("scale", np.nan))) + self.shape.setText(_field_text(data.get("shape", np.nan))) + self.minimum.setText(_field_text(data.get("minimum", np.nan))) + self.maximum.setText(_field_text(data.get("maximum", np.nan))) + self.neg_samples_cb.setChecked(bool(data.get("negative", False))) self._check_negative() + def _apply_session_sampled_to_widgets(self) -> None: + if self._session is None: + return + self._updating_from_pedigree = True + try: + self._write_sampled_fields(self._session.sampled) + self._sync_mean_from_loc() + finally: + self._updating_from_pedigree = False + + def _sync_pedigree_ui(self, *, sampled: bool = True) -> None: + if sampled: + self._apply_session_sampled_to_widgets() + self._apply_recipe_widgets(self._session.recipe) + using = self._session.use_pedigree + if self.use_pedigree is not None: + self.use_pedigree.blockSignals(True) + self.use_pedigree.setChecked(using) + self.use_pedigree.blockSignals(False) + if self.pedigree_box is not None: + self.pedigree_box.setVisible(using) + + def _on_use_pedigree_toggled(self, checked: bool) -> None: + if self._updating_from_pedigree or self._session is None: + return + if self._read_only: + self.pedigree_box.setVisible(checked) + return + if checked: + self._session.set_sampled(self._uncertainty_info) + self._session.check_use() + else: + self._session.uncheck_use() + self._sync_pedigree_ui() + self._schedule_plot_refresh() + + def _stop_using_pedigree_keep_sampled(self) -> None: + if self._session is None or not self._session.use_pedigree: + return + self._session.set_sampled(self._uncertainty_info) + self._session.stop_using_keep_sampled() + self._sync_pedigree_ui(sampled=False) + + def _on_pedigree_recipe_edited(self, *_args) -> None: + if ( + self._updating_from_pedigree + or self._session is None + or not self._session.use_pedigree + ): + return + self._session.edit_recipe(self._current_recipe()) + self._updating_from_pedigree = True + self.scale.setText(_field_text(self._session.sampled.get("scale"))) + self._updating_from_pedigree = False + self._schedule_plot_refresh() + + def _on_clear_pedigree(self) -> None: + if self._session is None or self._read_only: + return + self._session.clear() + self._sync_pedigree_ui() + self._schedule_plot_refresh() + + def _on_scale_edited(self) -> None: + if not self._updating_from_pedigree: + self._stop_using_pedigree_keep_sampled() + + def _pedigree_outcome(self) -> dict: + self._session.set_sampled(self._uncertainty_info) + if self._session.use_pedigree: + self._session.edit_recipe(self._current_recipe()) + outcome = self._session.outcome() + outcome["uncertainty"] = self._uncertainty_info + return outcome + def _apply_read_only_mode(self) -> None: self.setWindowTitle("View uncertainty") self.distribution.setEnabled(False) self.fields_box.setEnabled(False) + if self.pedigree_box is not None: + self.pedigree_box.setEnabled(False) ok_btn = self.buttons.button(QtWidgets.QDialogButtonBox.Ok) ok_btn.setVisible(False) ok_btn.setEnabled(False) @@ -269,6 +490,33 @@ def _apply_read_only_mode(self) -> None: cancel_btn.setText("Close") cancel_btn.setDefault(True) + def _available_dialog_height(self) -> int: + screen = self.screen() or QtWidgets.QApplication.primaryScreen() + if screen is None: + return 720 + return max(480, int(screen.availableGeometry().height()) - 72) + + def sizeHint(self) -> QtCore.QSize: + hint = super().sizeHint() + cap = self._available_dialog_height() + hint.setWidth(max(hint.width(), _DIALOG_OPEN_WIDTH)) + plot = getattr(self, "plot", None) + if plot is not None and plot.isVisible() and plot.maximumHeight() > 0: + hint.setHeight(min(max(hint.height(), _DIALOG_OPEN_HEIGHT), cap)) + else: + hint.setHeight(min(hint.height(), cap)) + return hint + + def showEvent(self, event: QtGui.QShowEvent) -> None: + super().showEvent(event) + self._clamp_dialog_to_screen() + + def _clamp_dialog_to_screen(self) -> None: + cap = self._available_dialog_height() + self.setMaximumHeight(cap) + if self.height() > cap: + self.resize(self.width(), cap) + @property def _field_widgets(self) -> dict[str, tuple[QtWidgets.QWidget, QtWidgets.QLineEdit]]: return { @@ -324,6 +572,13 @@ def _on_distribution_changed(self, index: int) -> None: self.dist.id not in (sa.UndefinedUncertainty.id, sa.NoUncertainty.id) ) self.previous_dist_id = self.dist.id + if ( + not self._updating_from_pedigree + and self._session is not None + and self._session.use_pedigree + and self.dist.id != sa.LognormalUncertainty.id + ): + self._stop_using_pedigree_keep_sampled() self._generate_plot() def _extract_lognormal_loc_from_mean(self) -> None: @@ -423,6 +678,12 @@ def _structured_array_if_sampleable(self) -> Tuple[Optional[np.ndarray], Optiona def _ok_enabled(self) -> bool: if self.dist is None: return False + if ( + self._session is not None + and self._session.use_pedigree + and not recipe_is_usable(self._current_recipe()) + ): + return False if self.dist.id in (sa.UndefinedUncertainty.id, sa.NoUncertainty.id): return True array, _ = self._structured_array_if_sampleable() @@ -453,6 +714,7 @@ def _relayout_dialog_compact(self) -> None: plot = getattr(self, "plot", None) if plot is None or plot.maximumHeight() <= 0 or not plot.isVisible(): self.adjustSize() + self._clamp_dialog_to_screen() def _hide_plot_preview(self, message: Optional[str]) -> None: """Hide the matplotlib preview, collapse its layout height, optional status text.""" @@ -530,8 +792,23 @@ def _on_accept(self) -> None: if not self._ok_enabled(): return try: - self.result_dict = self._uncertainty_info - self.result_array = UncertaintyBase.from_dicts(self._uncertainty_info) + info = self._uncertainty_info + if self._session is None: + self.result_dict = info + self.result_array = UncertaintyBase.from_dicts(info) + else: + result = resolve_pedigree_edit( + self._session.stored, self._pedigree_outcome() + ) + self.result_dict = dict(result.write) + for key in result.delete: + self.result_dict[key] = None + sampled = { + key: value + for key, value in self.result_dict.items() + if key != "pedigree" + } + self.result_array = UncertaintyBase.from_dicts(sampled) except Exception as e: QtWidgets.QMessageBox.warning( self, @@ -551,7 +828,7 @@ def __init__(self, parent=None): if hasattr(self.figure, "set_constrained_layout"): self.figure.set_constrained_layout(False) # Fixed floor for the preview strip (logical px); extra height helps x-axis label fit. - self.setMinimumHeight(348) + self.setMinimumHeight(_PLOT_MIN_HEIGHT) exp = QtWidgets.QSizePolicy.Policy.Expanding self.setSizePolicy(exp, exp) self.canvas.setSizePolicy(exp, exp) @@ -564,7 +841,7 @@ def plot_analytical( self, curve: PreviewDensity, vline_x: float, *, title: str = "" ) -> None: """Plot ``stats_arrays`` / SciPy PDF or PMF (no random sampling).""" - self.setMinimumHeight(348) + self.setMinimumHeight(_PLOT_MIN_HEIGHT) self.setMaximumHeight(16777215) self.setVisible(True) self.reset_plot() diff --git a/activity_browser/ui/widgets/tab_widget.py b/activity_browser/ui/widgets/tab_widget.py index dfbb33a9d..abc0b5abc 100644 --- a/activity_browser/ui/widgets/tab_widget.py +++ b/activity_browser/ui/widgets/tab_widget.py @@ -3,6 +3,14 @@ from .buttons import ABCloseButton, ABMinimizeButton +class ABTabBar(QtWidgets.QTabBar): + def minimumTabSizeHint(self, index): + # Must be a fixed floor — Qt's default min grows with the label, so + # max(super(), N) never kicks in and short names still get crushed. + s = super().minimumTabSizeHint(index) + return QtCore.QSize(80, s.height()) + + class ABTabWidget(QtWidgets.QTabWidget): def __init__(self, *args, **kwargs): """ @@ -13,6 +21,7 @@ def __init__(self, *args, **kwargs): *args: Additional positional arguments passed to the parent QTabWidget. """ super().__init__(*args, **kwargs) + self.setTabBar(ABTabBar()) self.setMovable(True) # Allow tabs to be rearranged. self.setTabsClosable(True) # Allow tabs to be closed. self.tabBar().setExpanding(False) diff --git a/docs/adr/0010-parameterized-flow-index.md b/docs/adr/0010-parameterized-flow-index.md new file mode 100644 index 000000000..fde7a4a0f --- /dev/null +++ b/docs/adr/0010-parameterized-flow-index.md @@ -0,0 +1,11 @@ +# Parameterized Flows follow Brightway’s `ParameterizedExchange` index + +The Parameters page **Parameterized Flows** table, recalculation, and Monte Carlo all use Brightway’s `ParameterizedExchange` index (activity-parameter group), not a scan of formula-bearing flows. A `bwutils` helper rebuilds that index for the **written** database on `on_database_write` when the database is small or already has parameters. **Small** means at most **1,000 outgoing flows**. Skip databases above that cap that have no database/activity parameters; otherwise delete that database’s index rows, index every process that has a formula-bearing flow (dummy activity parameter if needed), then recalculate those groups. A database over the cap that already has parameters still gets a full pass. Single-flow formula edits stay on ExchangeModify. No one-shot backfill of existing projects. + +Brightway `Database.write` sends `on_database_write` only when `projects.dataset.is_sourced` is true (default false), or when the caller passes `signal=True`. Excel import extra-sends that signal after bw2io writes with signals off. Database duplicate and BW25 migration pass `signal=True` so the same handler rebuilds the copy. + +## Considered Options + +- Scan formula-bearing flows on every Parameters-page sync (shows flows that will not recalculate). +- Fill the index only on Excel import via bw2io `activate_parameters` (misses database-parameter-only processes and other writes). +- One-shot backfill of open projects (rejected: user rebuilds those by rewrite). diff --git a/docs/adr/0011-pedigree-recipe-on-flows.md b/docs/adr/0011-pedigree-recipe-on-flows.md new file mode 100644 index 000000000..71044f74a --- /dev/null +++ b/docs/adr/0011-pedigree-recipe-on-flows.md @@ -0,0 +1,7 @@ +# Pedigree is a stored recipe on flows, applied opt-in + +Ecoinvent and Brightway store pedigree scores on a flow separately from the sampled uncertainty; Monte Carlo only samples the latter. Activity Browser treats pedigree as a stored recipe (five scores plus basic uncertainty) for lognormal spread. + +The uncertainty dialog hides the pedigree editor until **Use pedigree** is checked. Checking applies: lognormal spread from the recipe, persisted on OK. Unchecking restores the sampled fields from just before the check and leaves the stored recipe. Changing or removing the sampled uncertainty while using pedigree keeps that new choice, turns use pedigree off, and leaves the recipe. Clearing pedigree restores the sampled fields and deletes the stored recipe on OK; checking use pedigree again before OK undoes Clear. + +Pedigree is not offered on parameter objects or characterization factors. Parameterized flows still get pedigree because they are flows, not parameters. Pedigree is not restored as a wizard or a separate table column. diff --git a/docs/advanced-topics/scenario-calculations.md b/docs/advanced-topics/scenario-calculations.md index a94bf0211..e3c1335a8 100644 --- a/docs/advanced-topics/scenario-calculations.md +++ b/docs/advanced-topics/scenario-calculations.md @@ -32,7 +32,7 @@ When you import a flow scenario into a calculation setup, the Activity Browser w Flow scenario (SDF) files may include comments that are ignored on import: -- **Comment rows:** start the row with `#` (first cell, or a full CSV line). Handled by pandas `comment="#"`. +- **Comment rows:** start the row with `#` (first cell, or a full CSV line). CSV uses pandas `comment="#"`. Excel uses `skiprows` (pandas `comment="#"` breaks Excel headers under openpyxl). - **Comment columns:** give the column a name that starts with `_` (for example `_notes`). These are dropped via `usecols`. Do not start comment **column** names with `#` — that conflicts with pandas row comments and can corrupt the header. diff --git a/tests/actions/test_exchange_actions.py b/tests/actions/test_exchange_actions.py index 341b0ae17..2f90e5692 100644 --- a/tests/actions/test_exchange_actions.py +++ b/tests/actions/test_exchange_actions.py @@ -90,6 +90,25 @@ def test_exchange_modify(basic_database): assert exchange[0].amount == 200.0 +def test_exchange_modify_formula_indexes_process(basic_database): + from bw2data.parameters import ParameterizedExchange + + process = basic_database.get("process") + elementary = basic_database.get("elementary") + exchange = next( + exc for exc in process.exchanges() if exc.input == elementary + ) + + app.actions.ExchangeModify.run(exchange, {"formula": "6+6"}) + + from activity_browser.bwutils.commontasks import refresh_edge + + exchange = refresh_edge(exchange) + assert exchange["formula"] == "6+6" + assert exchange["amount"] == 12 + assert ParameterizedExchange.select().count() == 1 + + def test_exchange_new(basic_database): basic_database.new_node("other", type="processwithreferenceproduct", name="other_process").save() @@ -157,6 +176,100 @@ def test_exchange_uncertainty_modify(monkeypatch, basic_database): assert exchange[0]["negative"] == False +def test_exchange_uncertainty_modify_writes_pedigree(monkeypatch, basic_database): + process = basic_database.get("process") + elementary = basic_database.get("elementary") + exchange = [ + exc for exc in process.exchanges() if exc.input == elementary + ] + recipe = { + "reliability": 2, + "completeness": 1, + "temporal correlation": 1, + "geographical correlation": 1, + "further technological correlation": 1, + "basic uncertainty": 1.05, + } + mock_uncertainty = { + "uncertainty type": UniformUncertainty.id, + "loc": float("nan"), + "scale": float("nan"), + "shape": float("nan"), + "minimum": 5.0, + "maximum": 15.0, + "negative": False, + "pedigree": recipe, + } + monkeypatch.setattr( + UncertaintyDialog, + "get_uncertainty_dict", + lambda *args, **kwargs: (True, mock_uncertainty), + ) + app.actions.ExchangeUncertaintyModify.run(exchange) + assert exchange[0]["pedigree"]["reliability"] == 2 + assert exchange[0]["pedigree"]["basic uncertainty"] == 1.05 + + +def test_exchange_uncertainty_modify_without_pedigree_keeps_recipe(monkeypatch, basic_database): + process = basic_database.get("process") + elementary = basic_database.get("elementary") + exchange = [ + exc for exc in process.exchanges() if exc.input == elementary + ] + exchange[0]["pedigree"] = { + "reliability": 2, + "completeness": 1, + "temporal correlation": 1, + "geographical correlation": 1, + "further technological correlation": 1, + } + exchange[0].save() + mock_uncertainty = { + "uncertainty type": UniformUncertainty.id, + "loc": float("nan"), + "scale": float("nan"), + "shape": float("nan"), + "minimum": 5.0, + "maximum": 15.0, + "negative": False, + } + monkeypatch.setattr( + UncertaintyDialog, + "get_uncertainty_dict", + lambda *args, **kwargs: (True, mock_uncertainty), + ) + app.actions.ExchangeUncertaintyModify.run(exchange) + assert exchange[0]["pedigree"]["reliability"] == 2 + assert exchange[0]["minimum"] == 5.0 + + +def test_exchange_uncertainty_modify_clears_pedigree(monkeypatch, basic_database): + process = basic_database.get("process") + elementary = basic_database.get("elementary") + exchange = [ + exc for exc in process.exchanges() if exc.input == elementary + ] + exchange[0]["pedigree"] = { + "reliability": 3, + "completeness": 1, + "temporal correlation": 1, + "geographical correlation": 1, + "further technological correlation": 1, + } + exchange[0].save() + mock_uncertainty = { + "uncertainty type": UndefinedUncertainty.id, + "pedigree": None, + } + monkeypatch.setattr( + UncertaintyDialog, + "get_uncertainty_dict", + lambda *args, **kwargs: (True, mock_uncertainty), + ) + app.actions.ExchangeUncertaintyModify.run(exchange) + assert "pedigree" not in exchange[0] + + def test_exchange_uncertainty_remove(basic_database): process = basic_database.get("process") elementary = basic_database.get("elementary") @@ -173,3 +286,22 @@ def test_exchange_uncertainty_remove(basic_database): app.actions.ExchangeUncertaintyRemove.run(exchange) assert exchange[0].uncertainty_type == UndefinedUncertainty + + +def test_exchange_uncertainty_remove_keeps_pedigree(basic_database): + process = basic_database.get("process") + elementary = basic_database.get("elementary") + exchange = [ + exc for exc in process.exchanges() if exc.input == elementary + ] + exchange[0]["pedigree"] = { + "reliability": 2, + "completeness": 1, + "temporal correlation": 1, + "geographical correlation": 1, + "further technological correlation": 1, + } + exchange[0].save() + app.actions.ExchangeUncertaintyRemove.run(exchange) + assert exchange[0]["pedigree"]["reliability"] == 2 + assert exchange[0].uncertainty_type == UndefinedUncertainty diff --git a/tests/actions/test_project_actions.py b/tests/actions/test_project_actions.py index 1c710e032..7d0e87f03 100644 --- a/tests/actions/test_project_actions.py +++ b/tests/actions/test_project_actions.py @@ -1,3 +1,4 @@ +import sqlite3 from pathlib import Path from types import SimpleNamespace @@ -5,6 +6,7 @@ from bw2data import config from bw2data.parameters import ProjectParameter from bw2data.project import ProjectDataset +from bw2data.tests import bw2test from bw_processing import safe_filename from qtpy import QtWidgets @@ -32,6 +34,20 @@ def _open_sqlite_files_in(dir_path: Path) -> list[str]: return open_files +def _patch_delete_signals(monkeypatch): + import activity_browser.app.actions.project.project_delete as project_delete_mod + + monkeypatch.setattr( + project_delete_mod, + "app", + SimpleNamespace( + signals=SimpleNamespace( + project=SimpleNamespace(deleted=SimpleNamespace(emit=lambda *a, **k: None)) + ) + ), + ) + + def test_project_delete_closes_sqlite_before_removing_dir(monkeypatch, basic_database): """Windows cannot shutil.rmtree a project while parameters.db is still open.""" original = bd.projects.current @@ -54,15 +70,7 @@ def checking_rmtree(path, *args, **kwargs): return real_rmtree(path, *args, **kwargs) monkeypatch.setattr(project_delete_mod.shutil, "rmtree", checking_rmtree) - monkeypatch.setattr( - project_delete_mod, - "app", - SimpleNamespace( - signals=SimpleNamespace( - project=SimpleNamespace(deleted=SimpleNamespace(emit=lambda *a, **k: None)) - ) - ), - ) + _patch_delete_signals(monkeypatch) try: ProjectDelete.delete_project(victim, True) @@ -75,6 +83,31 @@ def checking_rmtree(path, *args, **kwargs): assert not dir_path.exists() +@bw2test +def test_project_delete_removes_dir_despite_leaked_parameters_connection(monkeypatch): + """WinError 32 when parameters.db is held outside peewee's SubstitutableDatabase.""" + _patch_delete_signals(monkeypatch) + + original = bd.projects.current + victim = "victim_leaked_sqlite_conn" + bd.projects.create_project(victim) + bd.projects.set_current(victim, update=False) + list(ProjectParameter.select()) + dir_path = _project_dir(victim) + leaked = sqlite3.connect(str(dir_path / "parameters.db")) + try: + ProjectDelete.delete_project(victim, True) + assert victim not in bd.projects + assert not dir_path.exists() + finally: + try: + leaked.close() + except Exception: + pass + if original in bd.projects: + bd.projects.set_current(original, update=False) + + def test_project_delete_run_removes_current_project(monkeypatch, basic_database): monkeypatch.setattr( ProjectDeletionDialog, "exec_", lambda self: ProjectDeletionDialog.Accepted diff --git a/tests/conftest.py b/tests/conftest.py index da4cd089e..f7458ed97 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -52,6 +52,9 @@ def _ensure_main_window() -> None: def _reset_main_window(qtbot) -> None: """Close extra tabs opened during a test; keep the main window alive.""" from activity_browser import app + from activity_browser.app.pages.activity_details.activity_details import ( + ActivityDetailsPage, + ) from activity_browser.ui import core qapp = QtWidgets.QApplication.instance() @@ -59,6 +62,11 @@ def _reset_main_window(qtbot) -> None: if mw is None or not core.qt_is_valid(mw): return + # Drop leftover Activity Details pages so delete signals cannot hit stale UI. + for page in list(mw.findChildren(ActivityDetailsPage)): + if core.qt_is_valid(page): + page.deleteLater() + central = mw.centralWidget() if central is not None and core.qt_is_valid(central): while central.count() > 1: @@ -70,7 +78,22 @@ def _reset_main_window(qtbot) -> None: if qapp is not None: qapp.processEvents(QtCore.QEventLoop.ProcessEventsFlag.AllEvents) - qtbot.wait(10) + + +@pytest.fixture(autouse=True) +def _sync_metadata_singleton(): + """Tests that replace MetaDataStore._instance must not desync app.metadata.""" + import sys + + yield + + app_module = sys.modules.get("activity_browser.app") + if app_module is None: + return + + from activity_browser.bwutils.metadata.metadata import MetaDataStore + + MetaDataStore._instance = app_module.metadata @pytest.fixture @@ -90,7 +113,9 @@ def main_window(qtbot, monkeypatch, no_exception_dialogs): _ensure_main_window() metadata.dataframe = pd.DataFrame() - app.main_window.show() + # show() is expensive on Windows offscreen; only raise if not already visible. + if not app.main_window.isVisible(): + app.main_window.show() yield app.main_window @@ -136,25 +161,46 @@ def mc_project(): yield CALCULATION_SETUP_NAME -@pytest.fixture +@pytest.fixture(scope="module") @bw2test def lcia_overview_project(): - """LCIA overview test database and calculation setups (1×1 … 10×10, MC).""" + """LCIA overview DB sized for current consumers (1×1 / 3×3), once per module. + + Full 10×10 fixture data remains available via ``fixtures.lcia_overview`` for + local/scripts use; CI tests only need the small setups. + """ from fixtures.lcia_overview import ( CALCULATION_SETUPS, DATABASE_NAME, - DATABASE, - METHODS, + build_database, + build_methods, ) - write_functional_database(DATABASE_NAME, DATABASE, process=True) - for method_key, cfs in METHODS.items(): + write_functional_database( + DATABASE_NAME, build_database(n_products=3), process=True + ) + for method_key, cfs in build_methods(n_methods=3).items(): write_method(method_key, cfs, process=True) - for cs_name, setup in CALCULATION_SETUPS.items(): - write_calculation_setup(cs_name, setup) + for cs_name in ("lcia_1x1", "lcia_3x3", "lcia_3x3_neg"): + write_calculation_setup(cs_name, CALCULATION_SETUPS[cs_name]) yield DATABASE_NAME +@pytest.fixture +@bw2test +def basic_project(): + """``basic`` DB + method + CS without main_window / metadata load. + + Prefer this over ``basic_database`` for pure Brightway / MLCA tests. + """ + from fixtures.basic import CALCULATION_SETUP, DATABASE, METHOD + + db = write_functional_database("basic", DATABASE, process=False, mark_dirty=True) + write_method("basic_method", METHOD, process=False) + write_calculation_setup("basic_calculation_setup", CALCULATION_SETUP) + yield db + + @pytest.fixture @bw2test def mc_project_with_parameters(): diff --git a/tests/fixtures/lcia_overview.py b/tests/fixtures/lcia_overview.py index 4c861afb7..b589c2673 100644 --- a/tests/fixtures/lcia_overview.py +++ b/tests/fixtures/lcia_overview.py @@ -79,10 +79,15 @@ def _main_process_key(index: int) -> tuple[str, str]: return (DATABASE_NAME, f"main_{index}") -def build_database(*, parameterize_prod_0_biosphere: bool = False) -> dict: +def build_database( + *, + n_products: int | None = None, + parameterize_prod_0_biosphere: bool = False, +) -> dict: data: dict = {} + n_products = N_PRODUCTS if n_products is None else n_products - for i in range(N_PRODUCTS): + for i in range(n_products): data[_elementary_key(i)] = { "name": f"elementary flow {i}", "code": f"elem_{i}", @@ -120,7 +125,7 @@ def build_database(*, parameterize_prod_0_biosphere: bool = False) -> dict: ], } - for i in range(N_PRODUCTS): + for i in range(n_products): pk = _product_data_key(i) mk = _main_process_key(i) data[pk] = { @@ -168,9 +173,10 @@ def build_database(*, parameterize_prod_0_biosphere: bool = False) -> dict: DATABASE_WITH_PARAMETER_FORMULA = build_database(parameterize_prod_0_biosphere=True) -def build_methods() -> dict[str, list]: +def build_methods(*, n_methods: int | None = None) -> dict[str, list]: methods = {} - for j in range(N_METHODS): + n_methods = N_METHODS if n_methods is None else n_methods + for j in range(n_methods): methods[method_name(j)] = [ ( _elementary_key(j), diff --git a/tests/test_calculation_setup_duplicate_fu.py b/tests/test_calculation_setup_duplicate_fu.py index 49ac09920..ecfdd07cd 100644 --- a/tests/test_calculation_setup_duplicate_fu.py +++ b/tests/test_calculation_setup_duplicate_fu.py @@ -22,7 +22,7 @@ def test_duplicate_reference_flow_in_cs_build_df(basic_database): assert df["process"].notna().all() -def test_duplicate_reference_flow_in_cs_calculate(basic_database): +def test_duplicate_reference_flow_in_cs_calculate(basic_project): cs_name = "basic_calculation_setup" key = ("basic", "product_1") cs = bd.calculation_setups[cs_name] @@ -47,7 +47,7 @@ def test_duplicate_reference_flow_in_cs_calculate(basic_database): assert overview["amount"].tolist() == [1.0, 2.0] -def test_duplicate_reference_flow_same_amount_inventory(basic_database): +def test_duplicate_reference_flow_same_amount_inventory(basic_project): """Same activity and amount twice must not collapse inventory columns.""" cs_name = "basic_calculation_setup" key = ("basic", "product_1") @@ -68,7 +68,7 @@ def test_duplicate_reference_flow_same_amount_inventory(basic_database): contributions.inventory_df(inventory_type="technosphere") -def test_setup_fu_labels_exclude_amount(basic_database): +def test_setup_fu_labels_exclude_amount(basic_project): from activity_browser.bwutils.multilca import _load_cs key = ("basic", "product_1") @@ -79,7 +79,7 @@ def test_setup_fu_labels_exclude_amount(basic_database): assert "1.0" not in obj.fu_labels[0] -def test_duplicate_reference_flow_contribution_columns(basic_database): +def test_duplicate_reference_flow_contribution_columns(basic_project): """Compare-by-method must keep one column per inv row, not collapse on label.""" cs_name = "basic_calculation_setup" key = ("basic", "product_1") @@ -103,7 +103,7 @@ def test_duplicate_reference_flow_contribution_columns(basic_database): assert set(numeric_cols) == {0, 1} -def test_superstructure_build_inventory_keeps_duplicate_reference_flows(basic_database): +def test_superstructure_build_inventory_keeps_duplicate_reference_flows(basic_project): """Scenario inventory table must have one column per inv row.""" from activity_browser.bwutils.multilca import MLCA from activity_browser.bwutils.superstructure.mlca import ( diff --git a/tests/test_database_roundtrip.py b/tests/test_database_roundtrip.py index d8b214a69..83c7ed739 100644 --- a/tests/test_database_roundtrip.py +++ b/tests/test_database_roundtrip.py @@ -8,7 +8,6 @@ from bw2data.tests import bw2test from bw2io import create_core_migrations, create_default_biosphere3 -from activity_browser.bwutils.metadata.loader import MDSLoader from activity_browser.bwutils.metadata.metadata import MetaDataStore from fixtures.database_roundtrip import roundtrip_import, visible_product_count, write_source_db @@ -18,30 +17,71 @@ def project_setup() -> None: create_default_biosphere3() +def _teardown_temporary_mds(mds: MetaDataStore) -> None: + """Stop loader threads and disconnect bw2data signals before dropping the instance.""" + from bw2data import signals + from bw2data.meta import databases + + loader = getattr(mds, "loader", None) + if loader is not None: + if loader.thread is not None and loader.thread.isRunning(): + loader.thread.wait(10_000) + try: + signals.project_changed.disconnect(loader.on_project_changed) + except (TypeError, RuntimeError): + pass + loader._disconnect_thread_results() + + updater = getattr(mds, "updater", None) + if updater is not None: + for signal, slot in ( + (signals.signaleddataset_on_save, updater.on_signaleddataset_save), + (signals.signaleddataset_on_delete, updater.on_signaleddataset_delete), + (signals.on_database_delete, updater.on_database_deleted_bw), + ): + try: + signal.disconnect(slot) + except (TypeError, RuntimeError): + pass + try: + databases._save_signal.disconnect(updater.on_databases_metadata_change) + except (TypeError, RuntimeError, AttributeError): + pass + + @bw2test def test_load_database_populates_metadata_for_excel_import(qapp, monkeypatch): monkeypatch.setattr( "activity_browser.bwutils.metadata.updater.MDSUpdater.connect_signals", lambda self: None, ) - MetaDataStore._instance = None - project_setup() - source = "roundtrip_metadata_src" - write_source_db(source, "functional") - - mds = MetaDataStore() - loader = MDSLoader(mds) - - with tempfile.TemporaryDirectory() as tmp: - target = roundtrip_import(source, "excel", tmp) - - assert mds.get_database_metadata(target, ["name"]).empty - loader.load_database(target) - for _ in range(100): - if loader.secondary_status == "done": - break - time.sleep(0.05) - qapp.processEvents() - - assert len(mds.get_database_metadata(target, ["name", "processor", "type"])) == 4 - assert visible_product_count(target) == 2 + previous_instance = MetaDataStore._instance + temp_mds = None + try: + MetaDataStore._instance = None + project_setup() + source = "roundtrip_metadata_src" + write_source_db(source, "functional") + + temp_mds = MetaDataStore() + # Use the loader owned by the store — a second MDSLoader would also + # connect to bw2data.project_changed and crash later @bw2test fixtures. + loader = temp_mds.loader + + with tempfile.TemporaryDirectory() as tmp: + target = roundtrip_import(source, "excel", tmp) + + assert temp_mds.get_database_metadata(target, ["name"]).empty + loader.load_database(target) + for _ in range(100): + if loader.secondary_status == "done": + break + time.sleep(0.05) + qapp.processEvents() + + assert len(temp_mds.get_database_metadata(target, ["name", "processor", "type"])) == 4 + assert visible_product_count(target) == 2 + finally: + if temp_mds is not None: + _teardown_temporary_mds(temp_mds) + MetaDataStore._instance = previous_instance diff --git a/tests/test_gsa.py b/tests/test_gsa.py index 93ea5107a..b0d9856f7 100644 --- a/tests/test_gsa.py +++ b/tests/test_gsa.py @@ -3,7 +3,9 @@ from __future__ import annotations import numpy as np +import pytest import stats_arrays as sa +from bw2data.tests import bw2test from activity_browser.bwutils.montecarlo import MonteCarloLCA from activity_browser.bwutils.sensitivity_analysis import ( @@ -16,16 +18,29 @@ get_lca, ) from activity_browser.app.pages.lca_results.plots import GSAPlot -from activity_browser.bwutils.sensitivity_analysis import GSA_NAME_COLUMN from activity_browser.bwutils.uncertainty import ( uncertainty_cell_summary, uncertainty_field_name, uncertainty_parameters_summary, ) -from fixtures.monte_carlo import CALCULATION_SETUP +from fixtures.bw_helpers import ( + register_parameter_setup, + write_calculation_setup, + write_functional_database, + write_method, +) +from fixtures.monte_carlo import ( + CALCULATION_SETUP, + CALCULATION_SETUP_NAME, + DATABASE_NAME, + DATABASE_WITH_PARAMETER_FORMULA, + METHOD, + METHOD_NAME, + PARAMETER_SETUP, +) -# SALib delta needs enough MC iterations; keep ≥ 40 for full multi-layer GSA. -ITERATIONS = 40 +# SALib delta needs enough MC iterations for a stable full multi-layer GSA. +ITERATIONS = 30 SEED = 42 ALL_UNCERTAINTY_LAYERS = dict( @@ -48,6 +63,79 @@ def _run_gsa(mc: MonteCarloLCA) -> GlobalSensitivityAnalysis: return gsa +@pytest.fixture(scope="module") +@bw2test +def gsa_mc_project(): + """One parameterized MC project for the whole module (shared MC/GSA runs).""" + write_functional_database("mc", DATABASE_WITH_PARAMETER_FORMULA, process=True) + register_parameter_setup(DATABASE_NAME, PARAMETER_SETUP) + write_method(METHOD_NAME, METHOD, process=True) + write_calculation_setup(CALCULATION_SETUP_NAME, CALCULATION_SETUP) + yield CALCULATION_SETUP_NAME + + +@pytest.fixture(scope="module") +def mc_all_layers(gsa_mc_project): + return _run_mc(gsa_mc_project, **ALL_UNCERTAINTY_LAYERS) + + +@pytest.fixture(scope="module") +def gsa_all_layers(mc_all_layers): + return _run_gsa(mc_all_layers) + + +@pytest.fixture(scope="module") +def mc_cf_only(gsa_mc_project): + return _run_mc( + gsa_mc_project, technosphere=False, biosphere=False, cf=True, parameters=False + ) + + +@pytest.fixture(scope="module") +def gsa_cf_only(mc_cf_only): + return _run_gsa(mc_cf_only) + + +@pytest.fixture(scope="module") +def mc_tech_bio(gsa_mc_project): + return _run_mc( + gsa_mc_project, technosphere=True, biosphere=True, cf=False, parameters=False + ) + + +@pytest.fixture(scope="module") +def gsa_tech_bio(mc_tech_bio): + return _run_gsa(mc_tech_bio) + + +@pytest.fixture(scope="module") +def mc_tech_only(gsa_mc_project): + return _run_mc( + gsa_mc_project, technosphere=True, biosphere=False, cf=False, parameters=False + ) + + +@pytest.fixture(scope="module") +def gsa_tech_only(mc_tech_only): + return _run_gsa(mc_tech_only) + + +@pytest.fixture(scope="module") +def mc_params_only(gsa_mc_project): + return _run_mc( + gsa_mc_project, + technosphere=False, + biosphere=False, + cf=False, + parameters=True, + ) + + +@pytest.fixture(scope="module") +def gsa_params_only(mc_params_only): + return _run_gsa(mc_params_only) + + def test_gsa_plot_renders_sample_dataframe(): import matplotlib @@ -55,8 +143,6 @@ def test_gsa_plot_renders_sample_dataframe(): import pandas as pd from qtpy import QtWidgets - from activity_browser.bwutils.sensitivity_analysis import GSA_TYPE_COLUMN - if QtWidgets.QApplication.instance() is None: QtWidgets.QApplication([]) @@ -100,7 +186,7 @@ def test_triangular_uncertainty_summary(): assert uncertainty_cell_summary(data) == "Triangular; Mode: 5.0; Minimum: 0.0; Maximum: 10.0" -def test_get_cf_dataframe_uses_method_uncertainty(mc_project): +def test_get_cf_dataframe_uses_method_uncertainty(gsa_mc_project): cs = CALCULATION_SETUP lca = get_lca(cs["inv"][0], cs["ia"][0]) dfcf, _ = get_CF_dataframe(lca, cs["ia"][0], only_uncertain_CFs=True) @@ -111,12 +197,12 @@ def test_get_cf_dataframe_uses_method_uncertainty(mc_project): assert "Minimum:" in dfcf.iloc[0]["uncertainty"] -def test_gsa_full_run_all_uncertainty_layers(mc_project_with_parameters): +def test_gsa_full_run_all_uncertainty_layers(mc_all_layers, gsa_all_layers): """End-to-end GSA with technosphere, biosphere, CF, and parameter MC uncertainty.""" - mc = _run_mc(mc_project_with_parameters, **ALL_UNCERTAINTY_LAYERS) + mc = mc_all_layers + gsa = gsa_all_layers assert mc.iterations == ITERATIONS - gsa = _run_gsa(mc) assert gsa.df_final is not None and not gsa.df_final.empty assert list(gsa.df_final.columns) == list(GSA_COLUMNS) @@ -132,18 +218,17 @@ def test_gsa_full_run_all_uncertainty_layers(mc_project_with_parameters): assert gsa.df_final["delta"].is_monotonic_decreasing -def test_gsa_runs_with_cf_uncertainty(mc_project): - mc = _run_mc(mc_project, technosphere=False, biosphere=False, cf=True, parameters=False) - gsa = _run_gsa(mc) +def test_gsa_runs_with_cf_uncertainty(gsa_cf_only): + gsa = gsa_cf_only assert gsa.df_final is not None and not gsa.df_final.empty assert (gsa.df_final[GSA_TYPE_COLUMN] == "characterization factor").any() assert list(gsa.df_final.columns) == list(GSA_COLUMNS) -def test_mc_matrix_snapshots_are_per_iteration(mc_project): +def test_mc_matrix_snapshots_are_per_iteration(mc_tech_bio, gsa_tech_bio): """Technosphere/biosphere draws must be copied each iteration, not shared references.""" - mc = _run_mc(mc_project, technosphere=True, biosphere=True, cf=False, parameters=False) + mc = mc_tech_bio assert len(mc.A_matrices) == ITERATIONS assert len(mc.B_matrices) == ITERATIONS @@ -156,7 +241,7 @@ def test_mc_matrix_snapshots_are_per_iteration(mc_project): assert len(a_snapshots) > 1 assert len(b_snapshots) > 1 - gsa = _run_gsa(mc) + gsa = gsa_tech_bio n_tech = len(gsa.t_indices) n_bio = len(gsa.b_indices) tech_X = gsa.X[:, :n_tech] @@ -165,9 +250,8 @@ def test_mc_matrix_snapshots_are_per_iteration(mc_project): assert not np.allclose(bio_X, bio_X[0]) -def test_exchange_gsa_name_format(mc_project): - mc = _run_mc(mc_project, technosphere=True, biosphere=False, cf=False, parameters=False) - gsa = _run_gsa(mc) +def test_exchange_gsa_name_format(gsa_tech_only): + gsa = gsa_tech_only tech = gsa.df_final.loc[gsa.df_final[GSA_TYPE_COLUMN] == "technosphere"].iloc[0] assert " --> " in tech[GSA_NAME_COLUMN] @@ -178,15 +262,8 @@ def test_exchange_gsa_name_format(mc_project): assert gsa.metadata.index.name == GSA_INDEX_COLUMN -def test_gsa_runs_with_parameter_uncertainty(mc_project_with_parameters): - mc = _run_mc( - mc_project_with_parameters, - technosphere=False, - biosphere=False, - cf=False, - parameters=True, - ) - gsa = _run_gsa(mc) +def test_gsa_runs_with_parameter_uncertainty(gsa_params_only): + gsa = gsa_params_only param = gsa.df_final.loc[gsa.df_final[GSA_TYPE_COLUMN] == "parameter"].iloc[0] assert "bio_amount" in param[GSA_NAME_COLUMN] @@ -195,18 +272,18 @@ def test_gsa_runs_with_parameter_uncertainty(mc_project_with_parameters): assert "Maximum: 12.0" in param["uncertainty"] -def test_gsa_export_basename(mc_project): +def test_gsa_export_basename(gsa_mc_project, gsa_all_layers): from activity_browser.bwutils.export_names import export_name_slug - gsa = _run_gsa(_run_mc(mc_project, **ALL_UNCERTAINTY_LAYERS)) + gsa = gsa_all_layers basename = gsa.get_save_name() - assert basename.startswith(f"{mc_project}_GSA_") + assert basename.startswith(f"{gsa_mc_project}_GSA_") assert export_name_slug(gsa.method) in basename assert not basename.endswith(".xlsx") assert "gsa_output" not in basename -def test_gsa_input_export_order_matches_output(mc_project): - gsa = _run_gsa(_run_mc(mc_project, **ALL_UNCERTAINTY_LAYERS)) +def test_gsa_input_export_order_matches_output(gsa_all_layers): + gsa = gsa_all_layers input_df = gsa._gsa_input_dataframe() assert input_df.index.tolist() == gsa.df_final[GSA_INDEX_COLUMN].tolist() diff --git a/tests/test_lca_inputs.py b/tests/test_lca_inputs.py index 696fb7bc7..4ae3279a6 100644 --- a/tests/test_lca_inputs.py +++ b/tests/test_lca_inputs.py @@ -6,11 +6,23 @@ import bw2data as bd import pytest +from bw2data.tests import bw2test from activity_browser.bwutils.lca_inputs import lca_for_tree_selection from activity_browser.bwutils.superstructure.manager import SuperstructureManager from activity_browser.bwutils.superstructure.mlca import SuperstructureMLCA -from fixtures.lcia_overview import build_scenario_dataframe +from fixtures.bw_helpers import ( + write_calculation_setup, + write_functional_database, + write_method, +) +from fixtures.lcia_overview import ( + CALCULATION_SETUPS, + DATABASE, + DATABASE_NAME, + METHODS, + build_scenario_dataframe, +) def _tree_demand(func_unit: dict) -> dict: @@ -18,15 +30,27 @@ def _tree_demand(func_unit: dict) -> dict: return {bd.get_activity(k).id: v for k, v in func_unit.items()} -def _scenario_mlca() -> SuperstructureMLCA: +@pytest.fixture(scope="module") +@bw2test +def tree_scenario_mlca(): + """One project + one SuperstructureMLCA.calculate for the whole module. + + Previously each test rebuilt ``lcia_overview_project`` and recalculated MLCA + (~7s each × 4 on Ubuntu CI). + """ + write_functional_database(DATABASE_NAME, DATABASE, process=True) + # lcia_1x1 only needs method_0; keep writes minimal. + write_method("lcia_method_0", METHODS["lcia_method_0"], process=True) + write_calculation_setup("lcia_1x1", CALCULATION_SETUPS["lcia_1x1"]) + df = SuperstructureManager(build_scenario_dataframe()).combined_data() mlca = SuperstructureMLCA("lcia_1x1", df) mlca.calculate() - return mlca + yield mlca -def test_tree_lca_scenarios_match_mlca_scores(lcia_overview_project): - mlca = _scenario_mlca() +def test_tree_lca_scenarios_match_mlca_scores(tree_scenario_mlca): + mlca = tree_scenario_mlca demand = _tree_demand(mlca.func_units[0]) method = mlca.methods[0] names = list(mlca.scenario_names) @@ -45,9 +69,9 @@ def test_tree_lca_scenarios_match_mlca_scores(lcia_overview_project): assert lca.score == pytest.approx(mlca.lca_scores[0, 0, idx]) -def test_tree_lca_without_scenarios_ignores_scenario_idx(lcia_overview_project): +def test_tree_lca_without_scenarios_ignores_scenario_idx(tree_scenario_mlca): """Regression: Tree used to always rebuild a non-scenario LCA after sankey update.""" - mlca = _scenario_mlca() + mlca = tree_scenario_mlca demand = _tree_demand(mlca.func_units[0]) method = mlca.methods[0] names = list(mlca.scenario_names) @@ -67,8 +91,8 @@ def test_tree_lca_without_scenarios_ignores_scenario_idx(lcia_overview_project): assert lca.score != pytest.approx(mlca.lca_scores[0, 0, high_idx]) -def test_tree_lca_switching_scenarios_updates_score(lcia_overview_project): - mlca = _scenario_mlca() +def test_tree_lca_switching_scenarios_updates_score(tree_scenario_mlca): + mlca = tree_scenario_mlca demand = _tree_demand(mlca.func_units[0]) method = mlca.methods[0] names = list(mlca.scenario_names) @@ -97,9 +121,9 @@ def test_tree_lca_switching_scenarios_updates_score(lcia_overview_project): assert low.score != pytest.approx(high_score) -def test_tree_lca_scenario_switch_does_not_warn_pardiso_noop(lcia_overview_project): +def test_tree_lca_scenario_switch_does_not_warn_pardiso_noop(tree_scenario_mlca): """``decompose_technosphere`` is a PARDISO no-op; do not call it on switch.""" - mlca = _scenario_mlca() + mlca = tree_scenario_mlca demand = _tree_demand(mlca.func_units[0]) method = mlca.methods[0] idx = list(mlca.scenario_names).index("high_demand") diff --git a/tests/test_lcia_overview.py b/tests/test_lcia_overview.py index ed68c74ea..be127ca49 100644 --- a/tests/test_lcia_overview.py +++ b/tests/test_lcia_overview.py @@ -106,6 +106,21 @@ def test_build_flows_x_methods_default_orientation(stub_method_units): assert data.values.shape == (2, 2) assert data.group_labels == ["m0", "m1"] assert data.series_labels[0].startswith("product ") + for col in ("amount", "unit", "product", "process", "location", "database"): + assert col in data.table_df.columns + assert "score unit" in data.table_df.columns + assert "relative" in data.table_df.columns + assert "value" not in data.table_df.columns + assert list(data.table_df.columns[:5]) == [ + "index", "series", "absolute", "relative", "score unit", + ] + assert list(data.table_df.columns[-6:]) == [ + "amount", "unit", "product", "process", "location", "database", + ] + assert data.table_df["database"].tolist() == ["db"] * 4 + assert list(data.table_df["amount"]) == [1, 1, 1, 1] + np.testing.assert_allclose(data.table_df["absolute"], [10.0, 5.0, 20.0, 15.0]) + np.testing.assert_allclose(data.table_df["relative"], [100.0, 50.0, 100.0, 75.0]) def test_build_flows_x_methods_flip_is_transpose_of_column_normalization(stub_method_units): @@ -168,6 +183,11 @@ def test_build_flows_x_scenarios_x_methods_panels(stub_method_units): ) assert len(data.panels) == 2 assert "impact category" in data.table_df.columns + for col in ("amount", "unit", "product", "process", "location", "database"): + assert col in data.table_df.columns + assert list(data.table_df.columns[-6:]) == [ + "amount", "unit", "product", "process", "location", "database", + ] def test_available_compare_modes_order(): @@ -194,7 +214,7 @@ def test_reference_flow_label_uses_processor_name(lcia_overview_project): import bw2data as bd from activity_browser.bwutils.commontasks import get_fu_label - from tests.fixtures.lcia_overview import DATABASE_NAME + from fixtures.lcia_overview import DATABASE_NAME act = bd.get_activity((DATABASE_NAME, "prod_0")) label = get_fu_label(act, 1.0) @@ -210,6 +230,15 @@ def test_lcia_1x1_calculation(lcia_overview_project): assert mlca.lca_scores[0, 0] > 0 +@pytest.fixture(scope="module") +def mlca_lcia_3x3(lcia_overview_project): + from activity_browser.bwutils.multilca import MLCA + + mlca = MLCA("lcia_3x3") + mlca.calculate() + return mlca + + def test_lcia_3x3_all_negative_column_normalization(lcia_overview_project): from activity_browser.bwutils.multilca import MLCA @@ -221,22 +250,20 @@ def test_lcia_3x3_all_negative_column_normalization(lcia_overview_project): assert np.isclose(normalized.min(), -100.0) -def test_lcia_overview_plot_smoke(lcia_overview_project): +def test_lcia_overview_plot_smoke(mlca_lcia_3x3): import matplotlib matplotlib.use("Agg") from qtpy import QtWidgets from activity_browser.app.pages.lca_results.plots import LCIAResultsOverviewPlot - from activity_browser.bwutils.multilca import MLCA, ca + from activity_browser.bwutils.multilca import ca if QtWidgets.QApplication.instance() is None: QtWidgets.QApplication([]) - mlca = MLCA("lcia_3x3") - mlca.calculate() data = build_lcia_overview( - mlca, + mlca_lcia_3x3, ca, compare=LCIACompareMode.FLOWS_X_METHODS, relative=True, @@ -246,6 +273,65 @@ def test_lcia_overview_plot_smoke(lcia_overview_project): assert len(plot.ax.patches) > 0 +def test_lcia_scores_table_reference_flow_columns(mlca_lcia_3x3): + from activity_browser.bwutils.multilca import Contributions + from fixtures.lcia_overview import DATABASE_NAME + + data = build_lcia_overview( + mlca_lcia_3x3, + Contributions(mlca_lcia_3x3), + compare=LCIACompareMode.REFERENCE_FLOWS, + relative=False, + method_index=0, + ) + df = data.table_df + assert list(df["product"]) == [f"product {i}" for i in range(3)] + assert list(df["process"]) == [f"main process {i}" for i in range(3)] + assert list(df["location"]) == ["GLO"] * 3 + assert list(df["database"]) == [DATABASE_NAME] * 3 + assert list(df["amount"]) == [1.0, 1.0, 1.0] + assert list(df["unit"]) == ["kg"] * 3 + assert "value" not in df.columns + assert df["score unit"].nunique() == 1 + assert list(df.columns[-6:]) == [ + "amount", "unit", "product", "process", "location", "database", + ] + np.testing.assert_allclose(df["absolute"], df["relative"] * df["absolute"].abs().max() / 100.0) + + +def test_lcia_scores_table_fu_columns_follow_series_when_not_flipped(stub_method_units): + scores = np.array([[10.0, 20.0], [5.0, 15.0]]) + mlca = _FakeMLCA( + scores, + func_units=[{("db", "a"): 1}, {("db", "b"): 2}], + methods=[("m0",), ("m1",)], + ) + data = build_lcia_overview( + mlca, + _FakeContributions(), + compare=LCIACompareMode.FLOWS_X_METHODS, + relative=False, + flip_groups=False, + ) + # groups = methods, series = FUs: (m0,a), (m0,b), (m1,a), (m1,b) + assert list(data.table_df["amount"]) == [1, 2, 1, 2] + assert list(data.table_df["database"]) == ["db", "db", "db", "db"] + np.testing.assert_allclose(data.table_df["absolute"], [10.0, 5.0, 20.0, 15.0]) + np.testing.assert_allclose(data.table_df["relative"], [100.0, 50.0, 100.0, 75.0]) + + flipped = build_lcia_overview( + mlca, + _FakeContributions(), + compare=LCIACompareMode.FLOWS_X_METHODS, + relative=False, + flip_groups=True, + ) + # groups = FUs, series = methods: (a,m0), (a,m1), (b,m0), (b,m1) + assert list(flipped.table_df["amount"]) == [1, 1, 2, 2] + np.testing.assert_allclose(flipped.table_df["absolute"], [10.0, 20.0, 5.0, 15.0]) + np.testing.assert_allclose(flipped.table_df["relative"], [100.0, 100.0, 50.0, 75.0]) + + def test_lcia_compare_label_helpers_round_trip(): from activity_browser.bwutils.lcia_overview import ( LCIA_COMPARE_LABELS, diff --git a/tests/test_metadata_singleton.py b/tests/test_metadata_singleton.py new file mode 100644 index 000000000..b4e1636c1 --- /dev/null +++ b/tests/test_metadata_singleton.py @@ -0,0 +1,13 @@ +"""Regression: MetaDataStore singleton must stay aligned with app.metadata.""" + +from activity_browser import app +from activity_browser.bwutils.metadata.metadata import MetaDataStore + + +def test_metadata_singleton_matches_app(basic_database): + assert MetaDataStore._instance is app.metadata + + process = basic_database.get("process") + meta = app.metadata.get_metadata([process.key], ["name", "product"]) + assert not meta.empty + assert meta.iloc[0]["name"] == process["name"] diff --git a/tests/test_monte_carlo_scenarios.py b/tests/test_monte_carlo_scenarios.py new file mode 100644 index 000000000..5532ca4e6 --- /dev/null +++ b/tests/test_monte_carlo_scenarios.py @@ -0,0 +1,326 @@ +""" +Primary-seam tests: Monte Carlo + scenario amounts on ``MonteCarloLCA.calculate``. +""" + +from __future__ import annotations + +import numpy as np +import pandas as pd +import bw2data as bd +from stats_arrays.distributions import GammaUncertainty + +from activity_browser.bwutils.montecarlo import MonteCarloLCA +from activity_browser.bwutils.montecarlo import build_overlay_from_df +from activity_browser.bwutils.multilca import databases_for_fu_keys +from activity_browser.bwutils.superstructure.manager import SuperstructureManager +from activity_browser.bwutils.superstructure.utils import SUPERSTRUCTURE +from fixtures.monte_carlo import ( + BASELINE_SCORE, + DATABASE_NAME, +) + +SEED = 42 +ITERATIONS = 3 + + +def _scenario_row(**fields) -> pd.DataFrame: + row = {c: np.nan for c in SUPERSTRUCTURE} + row.update(fields) + return SuperstructureManager(pd.DataFrame([row])).combined_data(skip_checks=True) + + +def _scenario_df_biosphere_on_main(amounts: dict[str, float]) -> pd.DataFrame: + if all(isinstance(v, float) and np.isnan(v) for v in amounts.values()): + idx = pd.MultiIndex.from_tuples( + [((DATABASE_NAME, "elementary"), (DATABASE_NAME, "main"), "biosphere")], + names=["input", "output", "flow"], + ) + return pd.DataFrame(amounts, index=idx) + + return _scenario_row( + **{ + "from database": DATABASE_NAME, + "from key": (DATABASE_NAME, "elementary"), + "to database": DATABASE_NAME, + "to key": (DATABASE_NAME, "main"), + "flow type": "biosphere", + **amounts, + } + ) + + +def _scenario_df_technosphere_on_main(amounts: dict[str, float]) -> pd.DataFrame: + return _scenario_row( + **{ + "from database": DATABASE_NAME, + "from key": (DATABASE_NAME, "supplier_product"), + "to database": DATABASE_NAME, + "to key": (DATABASE_NAME, "main"), + "flow type": "technosphere", + **amounts, + } + ) + + +def _run(cs_name: str, scenario_df=None, scenario=None, scenario_overlay=None, **includes) -> MonteCarloLCA: + mc = MonteCarloLCA(cs_name) + mc.calculate( + iterations=ITERATIONS, + seed=SEED, + scenario_df=scenario_df, + scenario=scenario, + scenario_overlay=scenario_overlay, + **includes, + ) + return mc + + +def _precomputed_overlay(cs_name: str, scenario_df, scenario: str): + prep = MonteCarloLCA(cs_name) + prep.lca = prep.construct_lca( + demands=prep.fu_demands, + method_config={"impact_categories": prep.methods}, + technosphere=False, + biosphere=False, + characterization=False, + ) + prep.lca.lci() + prep.lca.lcia() + return build_overlay_from_df( + prep.lca, + scenario_df, + databases_for_fu_keys(prep.fu_activity_keys), + scenario, + ) + + +def _scores(mc: MonteCarloLCA) -> np.ndarray: + return mc.results[:, 0, 0] + + +def test_mc_scenario_amounts_apply_when_uncertainty_off(mc_project): + """Deterministic MC follows the selected scenario amount (main bio 10 → 20).""" + sdf = _scenario_df_biosphere_on_main({"S1": 10.0, "S2": 20.0}) + s1 = _scores( + _run( + mc_project, + scenario_df=sdf, + scenario="S1", + technosphere=False, + biosphere=False, + cf=False, + parameters=False, + ) + ) + s2 = _scores( + _run( + mc_project, + scenario_df=sdf, + scenario="S2", + technosphere=False, + biosphere=False, + cf=False, + parameters=False, + ) + ) + np.testing.assert_allclose(s1, BASELINE_SCORE) + np.testing.assert_allclose(s2, 21.0) + + +def test_mc_scenario_by_index(mc_project): + sdf = _scenario_df_biosphere_on_main({"S1": 10.0, "S2": 20.0}) + scores = _scores( + _run( + mc_project, + scenario_df=sdf, + scenario=1, + technosphere=False, + biosphere=False, + cf=False, + parameters=False, + ) + ) + np.testing.assert_allclose(scores, 21.0) + + +def test_mc_scenario_technosphere_amount_applies(mc_project): + sdf = _scenario_df_technosphere_on_main({"S1": 2.0}) + scores = _scores( + _run( + mc_project, + scenario_df=sdf, + scenario="S1", + technosphere=False, + biosphere=False, + cf=False, + parameters=False, + ) + ) + np.testing.assert_allclose(scores, 12.0) + + +def test_mc_scenario_nan_keeps_database_amount(mc_project): + sdf = _scenario_df_biosphere_on_main({"S1": float("nan")}) + scores = _scores( + _run( + mc_project, + scenario_df=sdf, + scenario="S1", + technosphere=False, + biosphere=False, + cf=False, + parameters=False, + ) + ) + np.testing.assert_allclose(scores, BASELINE_SCORE) + + +def test_mc_scenario_uncertain_flow_keeps_database_uncertainty(mc_project): + sdf = _scenario_row( + **{ + "from database": DATABASE_NAME, + "from key": (DATABASE_NAME, "elementary"), + "to database": DATABASE_NAME, + "to key": (DATABASE_NAME, "supplier"), + "flow type": "biosphere", + "S2": 100.0, + } + ) + fixed = _scores( + _run( + mc_project, + scenario_df=sdf, + scenario="S2", + technosphere=False, + biosphere=False, + cf=False, + parameters=False, + ) + ) + stochastic = _scores( + _run( + mc_project, + scenario_df=sdf, + scenario="S2", + technosphere=False, + biosphere=True, + cf=False, + parameters=False, + ) + ) + np.testing.assert_allclose(fixed, 110.0) + assert stochastic.max() < 20.0 + assert np.std(stochastic) > 0 + + +def test_mc_scenario_gamma_uncertainty_keeps_database_sampling(mc_project): + """Gamma (and other non-deterministic types) defer to DB MC, not scenario pin.""" + supplier = bd.get_activity((DATABASE_NAME, "supplier")) + for exc in supplier.exchanges(): + if exc.get("type") == "biosphere": + exc.update( + { + "uncertainty type": GammaUncertainty.id, + "loc": 1.0, + "scale": 0.2, + "shape": 2.0, + "minimum": float("nan"), + "maximum": float("nan"), + "negative": False, + } + ) + exc.save() + bd.Database(DATABASE_NAME).process() + + sdf = _scenario_row( + **{ + "from database": DATABASE_NAME, + "from key": (DATABASE_NAME, "elementary"), + "to database": DATABASE_NAME, + "to key": (DATABASE_NAME, "supplier"), + "flow type": "biosphere", + "S2": 100.0, + } + ) + fixed = _scores( + _run( + mc_project, + scenario_df=sdf, + scenario="S2", + technosphere=False, + biosphere=False, + cf=False, + parameters=False, + ) + ) + stochastic = _scores( + _run( + mc_project, + scenario_df=sdf, + scenario="S2", + technosphere=False, + biosphere=True, + cf=False, + parameters=False, + ) + ) + np.testing.assert_allclose(fixed, 110.0) + assert stochastic.max() < 20.0 + assert np.std(stochastic) > 0 + + +def test_mc_parameters_win_over_scenario_amount(mc_project_with_parameters): + sdf = _scenario_df_biosphere_on_main({"S2": 100.0}) + scores = _scores( + _run( + mc_project_with_parameters, + scenario_df=sdf, + scenario="S2", + technosphere=False, + biosphere=False, + cf=False, + parameters=True, + ) + ) + assert scores.min() >= 8.5 + assert scores.max() <= 13.5 + + +def test_mc_scenario_with_precomputed_overlay(mc_project): + sdf = _scenario_df_biosphere_on_main({"S1": 10.0, "S2": 20.0}) + overlay = _precomputed_overlay(mc_project, sdf, "S2") + scores = _scores( + _run( + mc_project, + scenario_overlay=overlay, + technosphere=False, + biosphere=False, + cf=False, + parameters=False, + ) + ) + np.testing.assert_allclose(scores, 21.0) + + +def test_mc_last_run_summary(mc_project): + sdf = _scenario_df_biosphere_on_main({"S2": 20.0}) + mc = _run( + mc_project, + scenario_df=sdf, + scenario="S2", + technosphere=True, + biosphere=False, + cf=False, + parameters=False, + ) + assert mc.last_run_summary == { + "scenario": "S2", + "iterations": ITERATIONS, + "seed": SEED, + "includes": { + "technosphere": True, + "biosphere": False, + "cf": False, + "parameters": False, + }, + } diff --git a/tests/test_monte_carlo_uncertainty.py b/tests/test_monte_carlo_uncertainty.py index 07207d7ec..e5a43055c 100644 --- a/tests/test_monte_carlo_uncertainty.py +++ b/tests/test_monte_carlo_uncertainty.py @@ -9,7 +9,7 @@ from activity_browser.bwutils.montecarlo import MonteCarloLCA SEED = 42 -ITERATIONS = 20 +ITERATIONS = 3 def _run_mc(cs_name: str, **includes) -> MonteCarloLCA: diff --git a/tests/test_parameterized_exchanges.py b/tests/test_parameterized_exchanges.py new file mode 100644 index 000000000..845228931 --- /dev/null +++ b/tests/test_parameterized_exchanges.py @@ -0,0 +1,315 @@ +"""Rebuild Brightway parameterized-flow index; Parameterized Flows table follows the index.""" + +from bw2data.parameters import ActivityParameter, ParameterizedExchange +from bw2data.tests import bw2test + +import bw2data as bd + +from activity_browser.bwutils.parameters.formula_exchanges import ( + rebuild_parameterized_flow_index, +) +from fixtures.basic import DATABASE +from fixtures.bw_helpers import write_functional_database + + +@bw2test +def test_rebuild_indexes_small_database_constant_formula(): + write_functional_database("basic", DATABASE, process=True) + rebuild_parameterized_flow_index("basic") + + rows = list(ParameterizedExchange.select()) + assert [row.formula for row in rows] == ["5+5"] + assert ActivityParameter.select().where(ActivityParameter.database == "basic").count() >= 1 + process = bd.get_activity(("basic", "process")) + bio = next(exc for exc in process.exchanges() if exc.get("formula") == "5+5") + assert bio["amount"] == 10 + + +EV_CASE = { + ("ev", "elementary"): { + "name": "elementary", + "code": "elementary", + "unit": "kg", + "type": "emission", + "categories": ("air",), + }, + ("ev", "car_prod"): { + "name": "transport", + "code": "car_prod", + "location": "GLO", + "type": "product", + "unit": "km", + "processor": ("ev", "car"), + }, + ("ev", "car"): { + "name": "transport, passenger car, electric", + "code": "car", + "location": "GLO", + "type": "process", + "exchanges": [ + {"type": "production", "amount": 1, "input": ("ev", "car_prod")}, + { + "type": "biosphere", + "amount": 0.0, + "input": ("ev", "elementary"), + "formula": "battery_size", + }, + ], + }, +} + + +@bw2test +def test_rebuild_indexes_database_parameter_formulas_and_recalculates(): + write_functional_database("ev", EV_CASE, process=True) + bd.parameters.new_database_parameters( + [{"name": "battery_size", "amount": 0.00262, "formula": ""}], + "ev", + ) + ActivityParameter.delete().where(ActivityParameter.database == "ev").execute() + ParameterizedExchange.delete().execute() + assert ActivityParameter.select().count() == 0 + assert ParameterizedExchange.select().count() == 0 + + rebuild_parameterized_flow_index("ev") + + assert ActivityParameter.select().where(ActivityParameter.database == "ev").count() >= 1 + rows = list(ParameterizedExchange.select()) + assert [row.formula for row in rows] == ["battery_size"] + car = bd.get_activity(("ev", "car")) + bio = next(exc for exc in car.exchanges() if exc.get("formula") == "battery_size") + assert bio["amount"] == 0.00262 + + +@bw2test +def test_rebuild_leaves_existing_parameters_in_place(): + from bw2data.parameters import DatabaseParameter, ProjectParameter + + write_functional_database("ev", EV_CASE, process=True) + bd.parameters.new_project_parameters( + [{"name": "proj_k", "amount": 1.5, "formula": ""}] + ) + bd.parameters.new_database_parameters( + [{"name": "battery_size", "amount": 0.00262, "formula": ""}], + "ev", + ) + car = bd.get_activity(("ev", "car")) + bd.parameters.new_activity_parameters( + [{ + "name": "user_share", + "amount": 0.8, + "formula": "", + "database": "ev", + "code": "car", + }], + str(car.id), + ) + + rebuild_parameterized_flow_index("ev") + + assert ProjectParameter.get(name="proj_k").amount == 1.5 + assert DatabaseParameter.get(name="battery_size").amount == 0.00262 + assert ActivityParameter.get(name="user_share").amount == 0.8 + assert ParameterizedExchange.select().count() == 1 + + +@bw2test +def test_rebuild_skips_over_cap_without_parameters(monkeypatch): + monkeypatch.setattr( + "activity_browser.bwutils.parameters.formula_exchanges.INDEX_FLOW_CAP", + 0, + ) + write_functional_database("basic", DATABASE, process=True) + rebuild_parameterized_flow_index("basic") + assert ParameterizedExchange.select().count() == 0 + + +@bw2test +def test_rebuild_runs_over_cap_when_database_has_parameters(monkeypatch): + monkeypatch.setattr( + "activity_browser.bwutils.parameters.formula_exchanges.INDEX_FLOW_CAP", + 0, + ) + write_functional_database("ev", EV_CASE, process=True) + bd.parameters.new_database_parameters( + [{"name": "battery_size", "amount": 0.00262, "formula": ""}], + "ev", + ) + rebuild_parameterized_flow_index("ev") + assert ParameterizedExchange.select().count() == 1 + + +@bw2test +def test_rebuild_replaces_stale_index_ids_on_rewrite(): + from bw2data.backends import ExchangeDataset + + write_functional_database("basic", DATABASE, process=True) + rebuild_parameterized_flow_index("basic") + live = ParameterizedExchange.select().first() + ParameterizedExchange( + group=live.group, exchange=999_999_999, formula="stale" + ).save() + assert ParameterizedExchange.select().count() == 2 + + write_functional_database("basic", DATABASE, process=True) + rebuild_parameterized_flow_index("basic") + + ids = [row.exchange for row in ParameterizedExchange.select()] + assert 999_999_999 not in ids + assert len(ids) == 1 + exc = ExchangeDataset.get_by_id(ids[0]) + assert exc.output_database == "basic" + assert (exc.data or {}).get("formula") == "5+5" + + +@bw2test +def test_rebuild_indexes_production_and_technosphere_formulas(): + data = { + ("mix", "elementary"): { + "name": "elementary", + "code": "elementary", + "unit": "kg", + "type": "emission", + "categories": ("air",), + }, + ("mix", "steel_prod"): { + "name": "steel", + "code": "steel_prod", + "location": "GLO", + "type": "product", + "unit": "kg", + "processor": ("mix", "steel"), + }, + ("mix", "steel"): { + "name": "steel production", + "code": "steel", + "location": "GLO", + "type": "process", + "exchanges": [ + {"type": "production", "amount": 1, "input": ("mix", "steel_prod")}, + ], + }, + ("mix", "car_prod"): { + "name": "car", + "code": "car_prod", + "location": "GLO", + "type": "product", + "unit": "kg", + "processor": ("mix", "car"), + }, + ("mix", "car"): { + "name": "car production", + "code": "car", + "location": "GLO", + "type": "process", + "exchanges": [ + { + "type": "production", + "amount": 0, + "input": ("mix", "car_prod"), + "formula": "1+1", + }, + { + "type": "technosphere", + "amount": 0, + "input": ("mix", "steel_prod"), + "formula": "3+4", + }, + { + "type": "biosphere", + "amount": 0, + "input": ("mix", "elementary"), + "formula": "5+5", + }, + ], + }, + } + write_functional_database("mix", data, process=True) + rebuild_parameterized_flow_index("mix") + + formulas = sorted(row.formula for row in ParameterizedExchange.select()) + assert formulas == ["1+1", "3+4", "5+5"] + car = bd.get_activity(("mix", "car")) + amounts = { + exc.get("formula"): exc["amount"] + for exc in car.exchanges() + if exc.get("formula") + } + assert amounts == {"1+1": 2, "3+4": 7, "5+5": 10} + + +def _relabel_database(data: dict, old: str, new: str) -> dict: + relabeled = {} + for (db, code), ds in data.items(): + copied = dict(ds) + copied["code"] = code + if copied.get("processor"): + proc_db, proc_code = copied["processor"] + copied["processor"] = (new if proc_db == old else proc_db, proc_code) + exchanges = [] + for exc in copied.get("exchanges", []): + row = dict(exc) + if row.get("input"): + in_db, in_code = row["input"] + row["input"] = (new if in_db == old else in_db, in_code) + exchanges.append(row) + if exchanges: + copied["exchanges"] = exchanges + relabeled[(new if db == old else db, code)] = copied + return relabeled + + +@bw2test +def test_rebuild_keeps_index_rows_from_other_databases(): + write_functional_database("basic", DATABASE, process=True) + rebuild_parameterized_flow_index("basic") + write_functional_database("other", _relabel_database(DATABASE, "basic", "other"), process=True) + rebuild_parameterized_flow_index("other") + + formulas_by_db = {} + for row in ParameterizedExchange.select(): + from bw2data.backends import ExchangeDataset + + exc = ExchangeDataset.get_by_id(row.exchange) + formulas_by_db.setdefault(exc.output_database, []).append(row.formula) + + assert formulas_by_db == {"basic": ["5+5"], "other": ["5+5"]} + + +@bw2test +def test_excel_importer_extra_sends_database_write(monkeypatch): + from types import SimpleNamespace + + from activity_browser.bwutils.importers import ABExcelImporter + from bw2data.signals import on_database_write + + received = [] + on_database_write.connect( + lambda sender, name: received.append(name), + weak=False, + ) + monkeypatch.setattr( + "bw2io.importers.base_lci.LCIImporter.write_database", + lambda self, **kwargs: SimpleNamespace(name="excel_db"), + ) + importer = ABExcelImporter.__new__(ABExcelImporter) + result = ABExcelImporter.write_database(importer) + + assert received == ["excel_db"] + assert result.name == "excel_db" + + +@bw2test +def test_parameters_in_scope_returns_empty_when_database_locked(monkeypatch): + from peewee import OperationalError + + from activity_browser.bwutils.commontasks import parameters_in_scope + + def locked(*args, **kwargs): + raise OperationalError("database is locked") + + monkeypatch.setattr( + "activity_browser.bwutils.commontasks.refresh_node", + locked, + ) + assert parameters_in_scope(node=("basic", "process")) == {} diff --git a/tests/test_parameterized_flows_table.py b/tests/test_parameterized_flows_table.py new file mode 100644 index 000000000..efabba684 --- /dev/null +++ b/tests/test_parameterized_flows_table.py @@ -0,0 +1,77 @@ +"""Parameterized Flows table follows Brightway's parameterized-flow index.""" + +from activity_browser import app # noqa: F401 — ABApplication before pytest-qt's QApplication +from bw2data.parameters import ParameterizedExchange + +from activity_browser.bwutils.parameters.formula_exchanges import ( + rebuild_parameterized_flow_index, +) + + +def test_parameterized_flows_table_follows_index_only(basic_database): + from activity_browser.app.pages.parameters.parameterized_exchanges_section import ( + ParameterizedExchangesSection, + ) + + section = ParameterizedExchangesSection() + ParameterizedExchange.delete().execute() + assert len(section.build_exchanges_df()) == 0 + + rebuild_parameterized_flow_index("basic") + df = section.build_exchanges_df() + assert len(df) == 1 + assert df.iloc[0]["formula"] == "5+5" + assert df.iloc[0]["_exchange"] is not None + + +def test_app_database_write_handler_rebuilds_index(basic_database): + from copy import deepcopy + + from bw2data.parameters import ParameterizedExchange + from fixtures.basic import DATABASE + + ParameterizedExchange.delete().execute() + basic_database.write(deepcopy(DATABASE), process=True, signal=True) + assert [row.formula for row in ParameterizedExchange.select()] == ["5+5"] + + +def test_formula_delegate_paint_survives_locked_database(basic_database, monkeypatch, qtbot): + from peewee import OperationalError + from qtpy import QtCore, QtGui, QtWidgets + + from activity_browser.app.pages.parameters.parameterized_exchanges_section import ( + ParameterizedExchangesSection, + ) + + rebuild_parameterized_flow_index("basic") + section = ParameterizedExchangesSection() + qtbot.addWidget(section) + section.sync() + + def locked(*args, **kwargs): + raise OperationalError("database is locked") + + monkeypatch.setattr( + "activity_browser.bwutils.commontasks.refresh_node", + locked, + ) + + model = section.model + formula_col = model.columns().index("formula") + index = model.index(0, formula_col) + assert index.isValid() + + from activity_browser.ui.delegates.new_formula import NewFormulaDelegate + + delegate = NewFormulaDelegate(section.view) + option = QtWidgets.QStyleOptionViewItem() + option.rect = QtCore.QRect(0, 0, 120, 24) + option.state = QtWidgets.QStyle.State_None + option.palette = section.view.palette() + + image = QtGui.QImage(120, 24, QtGui.QImage.Format_ARGB32) + painter = QtGui.QPainter(image) + try: + delegate.paint(painter, option, index) + finally: + painter.end() diff --git a/tests/test_pedigree.py b/tests/test_pedigree.py new file mode 100644 index 000000000..b972e5591 --- /dev/null +++ b/tests/test_pedigree.py @@ -0,0 +1,374 @@ +"""Pedigree recipe: infer basic uncertainty and resolve what to persist on a flow.""" +import math + +import pytest +import stats_arrays as sa + +from activity_browser.bwutils.pedigree import ( + PedigreeEditSession, + infer_basic_uncertainty, + resolve_pedigree_edit, +) +from activity_browser.bwutils.uncertainty import EMPTY_UNCERTAINTY + + +def test_infer_basic_uncertainty_all_perfect_scores(): + """All scores 1 contribute no extra spread; basic = exp(2 * scale).""" + recipe = { + "reliability": 1, + "completeness": 1, + "temporal correlation": 1, + "geographical correlation": 1, + "further technological correlation": 1, + } + assert infer_basic_uncertainty(recipe, scale=0.2) == pytest.approx(math.exp(0.4)) + + +def test_infer_basic_uncertainty_none_when_scale_tighter_than_scores(): + recipe = { + "reliability": 5, + "completeness": 5, + "temporal correlation": 5, + "geographical correlation": 5, + "further technological correlation": 5, + } + assert infer_basic_uncertainty(recipe, scale=0.001) is None + + +def test_incomplete_recipe_is_not_usable(): + from activity_browser.bwutils.pedigree import recipe_is_usable + + assert not recipe_is_usable({"reliability": 2}) + assert not recipe_is_usable(None) + + +def _perfect_recipe(): + return { + "reliability": 1, + "completeness": 1, + "temporal correlation": 1, + "geographical correlation": 1, + "further technological correlation": 1, + } + + +def _lognormal(loc=1.0, scale=0.2): + return { + **EMPTY_UNCERTAINTY, + "uncertainty type": sa.LognormalUncertainty.id, + "loc": loc, + "scale": scale, + } + + +def test_resolve_not_applying_untouched_does_not_write_pedigree(): + stored = _perfect_recipe() + outcome = { + "uncertainty": _lognormal(loc=1.1, scale=0.2), + "pedigree_applying": False, + "recipe_cleared": False, + "recipe": {**stored, "basic uncertainty": math.exp(0.4)}, + } + result = resolve_pedigree_edit(stored, outcome) + assert "pedigree" not in result.write + assert result.delete == () + assert result.write["loc"] == 1.1 + + +def test_resolve_not_applying_never_writes_pedigree(): + stored = _perfect_recipe() + recipe = {**stored, "reliability": 3, "basic uncertainty": 1.05} + outcome = { + "uncertainty": _lognormal(), + "pedigree_applying": False, + "recipe_cleared": False, + "recipe": recipe, + } + result = resolve_pedigree_edit(stored, outcome) + assert "pedigree" not in result.write + assert result.delete == () + + +def test_resolve_applying_keeps_imported_sample_size(): + stored = {**_perfect_recipe(), "sample size": 1} + recipe = {**_perfect_recipe(), "reliability": 3, "basic uncertainty": 1.0} + outcome = { + "uncertainty": _lognormal(), + "pedigree_applying": True, + "recipe_cleared": False, + "recipe": recipe, + } + result = resolve_pedigree_edit(stored, outcome) + assert result.write["pedigree"]["sample size"] == 1 + assert result.write["pedigree"]["reliability"] == 3 + + +def test_resolve_applying_sets_lognormal_scale_from_recipe(): + recipe = {**_perfect_recipe(), "reliability": 2, "basic uncertainty": 1.0} + outcome = { + "uncertainty": _lognormal(loc=0.5, scale=9.0), + "pedigree_applying": True, + "recipe_cleared": False, + "recipe": recipe, + } + result = resolve_pedigree_edit(_perfect_recipe(), outcome) + assert result.write["uncertainty type"] == sa.LognormalUncertainty.id + assert result.write["loc"] == 0.5 + assert result.write["scale"] == pytest.approx(math.log(1.54) / 2) + assert result.write["pedigree"]["reliability"] == 2 + assert result.write["pedigree"]["basic uncertainty"] == pytest.approx(1.0) + + +def test_pedigree_factors_tuple_omits_basic_uncertainty(): + from activity_browser.bwutils.pedigree import PedigreeMatrix + + matrix = PedigreeMatrix.from_dict({**_perfect_recipe(), "basic uncertainty": 1.5}) + assert 1.5 not in matrix.factors_as_tuple() + assert matrix.factors_as_tuple()[:5] == (1, 1, 1, 1, 1) + + +def test_resolve_cleared_deletes_pedigree_and_keeps_sampled_uncertainty(): + outcome = { + "uncertainty": _lognormal(loc=0.3, scale=0.1), + "pedigree_applying": False, + "recipe_cleared": True, + "recipe": None, + } + result = resolve_pedigree_edit(_perfect_recipe(), outcome) + assert result.delete == ("pedigree",) + assert "pedigree" not in result.write + assert result.write["loc"] == 0.3 + + +def test_resolve_applying_without_recipe_is_invalid(): + outcome = { + "uncertainty": _lognormal(), + "pedigree_applying": True, + "recipe_cleared": False, + "recipe": None, + } + with pytest.raises(ValueError): + resolve_pedigree_edit(None, outcome) + + +def test_from_dict_ignores_basic_uncertainty_key(): + from activity_browser.bwutils.pedigree import PedigreeMatrix + + matrix = PedigreeMatrix.from_dict({**_perfect_recipe(), "basic uncertainty": 1.5}) + assert "basic uncertainty" not in matrix.factors + + +def test_uncertainty_cell_includes_pedigree_scores_not_basic(): + from activity_browser.bwutils.uncertainty import uncertainty_cell_summary + + sampled = _lognormal(loc=0.0, scale=0.2) + recipe = {**_perfect_recipe(), "reliability": 2, "basic uncertainty": 1.5} + text = uncertainty_cell_summary(sampled, pedigree=recipe) + assert text.startswith("Lognormal") + assert "pedigree: 2, 1, 1, 1, 1" in text + assert "1.5" not in text + + +def test_uncertainty_cell_shows_pedigree_when_distribution_is_not_lognormal(): + from activity_browser.bwutils.uncertainty import uncertainty_cell_summary + + sampled = { + **EMPTY_UNCERTAINTY, + "uncertainty type": sa.UniformUncertainty.id, + "minimum": 0.0, + "maximum": 1.0, + } + text = uncertainty_cell_summary(sampled, pedigree=_perfect_recipe()) + assert "Uniform" in text + assert "pedigree: 1, 1, 1, 1, 1" in text + + +def test_display_basic_uncertainty_stored_wins_over_infer(): + from activity_browser.bwutils.pedigree import display_basic_uncertainty + + recipe = {**_perfect_recipe(), "basic uncertainty": 1.2} + assert display_basic_uncertainty( + recipe, scale=0.2, uncertainty_type=sa.LognormalUncertainty.id + ) == pytest.approx(1.2) + + +def test_display_basic_uncertainty_infers_when_unset(): + from activity_browser.bwutils.pedigree import display_basic_uncertainty + + assert display_basic_uncertainty( + _perfect_recipe(), scale=0.2, uncertainty_type=sa.LognormalUncertainty.id + ) == pytest.approx(math.exp(0.4)) + + +def _uniform(): + return { + **EMPTY_UNCERTAINTY, + "uncertainty type": sa.UniformUncertainty.id, + "minimum": 0.0, + "maximum": 1.0, + } + + +def test_session_opens_not_using_pedigree(): + stored = _perfect_recipe() + session = PedigreeEditSession(_uniform(), stored) + assert session.use_pedigree is False + assert session.recipe_cleared is False + assert session.sampled["uncertainty type"] == sa.UniformUncertainty.id + outcome = session.outcome() + assert outcome["pedigree_applying"] is False + assert outcome["recipe_cleared"] is False + + +def test_session_check_forces_lognormal_scale_from_recipe(): + stored = {**_perfect_recipe(), "reliability": 2} + session = PedigreeEditSession(_uniform(), stored) + session.check_use() + assert session.use_pedigree is True + assert session.sampled["uncertainty type"] == sa.LognormalUncertainty.id + assert session.sampled["scale"] == pytest.approx(math.log(1.54) / 2) + assert session.sampled["minimum"] == 0.0 + outcome = session.outcome() + assert outcome["pedigree_applying"] is True + assert outcome["recipe"]["reliability"] == 2 + + +def test_session_uncheck_restores_negative_flag(): + sampled = { + **EMPTY_UNCERTAINTY, + "uncertainty type": sa.GammaUncertainty.id, + "shape": 2.0, + "scale": 1.0, + "loc": 0.0, + "negative": True, + } + session = PedigreeEditSession(sampled, _perfect_recipe()) + session.check_use() + session.uncheck_use() + assert session.sampled["negative"] is True + assert session.sampled["uncertainty type"] == sa.GammaUncertainty.id + + +def test_session_uncheck_restores_pre_check_sampled_fields(): + session = PedigreeEditSession(_uniform(), _perfect_recipe()) + session.check_use() + session.uncheck_use() + assert session.use_pedigree is False + assert session.sampled["uncertainty type"] == sa.UniformUncertainty.id + assert session.sampled["minimum"] == 0.0 + assert session.sampled["maximum"] == 1.0 + + +def test_session_uncheck_discards_score_edits(): + stored = {**_perfect_recipe(), "reliability": 2} + session = PedigreeEditSession(_uniform(), stored) + session.check_use() + session.edit_recipe({**session.recipe, "reliability": 5}) + session.uncheck_use() + session.check_use() + assert session.recipe["reliability"] == 2 + + +def test_session_distribution_change_keeps_new_type(): + session = PedigreeEditSession(_lognormal(), _perfect_recipe()) + session.check_use() + session.set_sampled(_uniform()) + session.stop_using_keep_sampled() + assert session.use_pedigree is False + assert session.sampled["uncertainty type"] == sa.UniformUncertainty.id + assert session.sampled["minimum"] == 0.0 + assert session.outcome()["recipe_cleared"] is False + + +def test_session_scale_edit_keeps_new_scale(): + session = PedigreeEditSession(_lognormal(loc=0.5, scale=0.2), _perfect_recipe()) + session.check_use() + session.set_sampled(_lognormal(loc=0.5, scale=0.9)) + session.stop_using_keep_sampled() + assert session.use_pedigree is False + assert session.sampled["scale"] == pytest.approx(0.9) + assert session.sampled["loc"] == pytest.approx(0.5) + + +def test_session_clear_restores_and_marks_delete(): + session = PedigreeEditSession(_uniform(), _perfect_recipe()) + session.check_use() + session.clear() + assert session.use_pedigree is False + assert session.recipe_cleared is True + assert session.sampled["uncertainty type"] == sa.UniformUncertainty.id + outcome = session.outcome() + assert outcome["recipe_cleared"] is True + assert outcome["pedigree_applying"] is False + assert outcome["recipe"] is None + + +def test_session_check_after_clear_restores_stored_recipe(): + stored = {**_perfect_recipe(), "reliability": 4} + session = PedigreeEditSession(_uniform(), stored) + session.check_use() + session.clear() + session.check_use() + assert session.recipe_cleared is False + assert session.recipe["reliability"] == 4 + assert session.use_pedigree is True + + +def test_session_no_stored_recipe_defaults_scores_and_basic_to_one(): + session = PedigreeEditSession(_uniform(), None) + session.check_use() + assert session.recipe["reliability"] == 1 + assert session.recipe["completeness"] == 1 + assert session.recipe["basic uncertainty"] == pytest.approx(1.0) + assert session.sampled["scale"] == pytest.approx(0.0) + + +def test_session_malformed_stored_recipe_uses_defaults(): + session = PedigreeEditSession(_uniform(), {"reliability": 2}) + session.check_use() + assert session.recipe["reliability"] == 1 + assert session.recipe["basic uncertainty"] == pytest.approx(1.0) + + +def test_session_loc_edit_while_using_does_not_stop(): + session = PedigreeEditSession(_lognormal(loc=0.5, scale=0.2), _perfect_recipe()) + session.check_use() + session.set_sampled({**session.sampled, "loc": 1.2}) + assert session.use_pedigree is True + assert session.sampled["loc"] == pytest.approx(1.2) + + +def test_session_stored_basic_used_when_checking(): + stored = {**_perfect_recipe(), "basic uncertainty": 1.2} + session = PedigreeEditSession(_lognormal(), stored) + session.check_use() + assert session.recipe["basic uncertainty"] == pytest.approx(1.2) + + +def test_session_infers_basic_when_checking_lognormal_without_stored_basic(): + stored = _perfect_recipe() + session = PedigreeEditSession(_lognormal(loc=1.0, scale=0.2), stored) + session.check_use() + assert session.recipe["basic uncertainty"] == pytest.approx(math.exp(0.4)) + assert session.sampled["scale"] == pytest.approx(0.2) + + +def test_session_check_outcome_resolver_writes_recipe(): + stored = {**_perfect_recipe(), "reliability": 2} + session = PedigreeEditSession(_uniform(), stored) + session.check_use() + result = resolve_pedigree_edit(stored, session.outcome()) + assert result.write["uncertainty type"] == sa.LognormalUncertainty.id + assert result.write["pedigree"]["reliability"] == 2 + assert result.delete == () + + +def test_session_uncheck_outcome_resolver_does_not_write_pedigree(): + stored = _perfect_recipe() + session = PedigreeEditSession(_uniform(), stored) + session.check_use() + session.uncheck_use() + result = resolve_pedigree_edit(stored, session.outcome()) + assert "pedigree" not in result.write + assert result.delete == () + assert result.write["uncertainty type"] == sa.UniformUncertainty.id diff --git a/tests/test_reference_flow_labels.py b/tests/test_reference_flow_labels.py index 9dcf678ce..9e9c69e3e 100644 --- a/tests/test_reference_flow_labels.py +++ b/tests/test_reference_flow_labels.py @@ -6,6 +6,7 @@ from activity_browser.bwutils.commontasks import get_fu_label, get_method_label, reference_flow_parts from activity_browser.bwutils.contribution_labels import ( + apply_contribution_column_labels, contribution_column_labels, contribution_row_labels, ) @@ -234,6 +235,74 @@ def currentIndex(self): assert contribution_column_labels(tab, [0, 1]) == ["m, a", "long, name"] +def test_contribution_table_headers_show_impact_category_names(product_activity, monkeypatch): + """EF / process contribution tables must show IC names, not setup indices.""" + monkeypatch.setattr( + "activity_browser.bwutils.multilca.bd.get_activity", lambda key: product_activity + ) + key = ("LCIA_overview_test", "prod_0") + mlca = type("MLCA", (), {})() + _load_cs(mlca, [{key: 1.0}], [("m", "a"), ("long", "name")]) + + class Switches: + indexes = type("I", (), {"func": 0, "method": 1, "scenario": 2})() + mode = 1 + + def currentIndex(self): + return self.mode + + tab = type("Tab", (), {"switches": Switches(), "parent": type("P", (), {"mlca": mlca})()})() + df = pd.DataFrame( + { + "index": ["Score", "Rest (+)", "CO2"], + "name": ["Score", "Rest (+)", "Carbon dioxide"], + "unit": ["", "", "kg CO2-eq"], + 0: [10.0, 1.0, 9.0], + 1: [8.0, 0.5, 7.5], + } + ) + labelled = apply_contribution_column_labels(df, tab) + assert list(labelled.columns[:3]) == ["index", "name", "unit"] + assert list(labelled.columns[-2:]) == ["m, a", "long, name"] + assert 0 not in list(labelled.columns) + assert 1 not in list(labelled.columns) + + +def test_apply_contribution_column_labels_keeps_duplicate_fu_columns(product_activity, monkeypatch): + """Same reference-flow label twice must stay two table columns.""" + import numpy as np + + from activity_browser.bwutils.multilca import setup_index + + monkeypatch.setattr( + "activity_browser.bwutils.multilca.bd.get_activity", lambda key: product_activity + ) + key = ("LCIA_overview_test", "prod_0") + mlca = type("MLCA", (), {})() + _load_cs(mlca, [{key: 1.0}, {key: 2.0}], [("m", "a")]) + assert mlca.fu_labels[0] == mlca.fu_labels[1] + assert setup_index(np.int64(0)) == 0 + + class Switches: + indexes = type("I", (), {"func": 0, "method": 1, "scenario": 2})() + mode = 0 + + def currentIndex(self): + return self.mode + + tab = type("Tab", (), {"switches": Switches(), "parent": type("P", (), {"mlca": mlca})()})() + df = pd.DataFrame( + { + "index": ["Score"], + np.int64(0): [1.0], + np.int64(1): [2.0], + } + ) + labelled = apply_contribution_column_labels(df, tab) + assert list(labelled.columns[1:]) == [mlca.fu_labels[0], mlca.fu_labels[1]] + assert labelled.shape[1] == 3 + + def test_join_df_with_metadata_reference_flow_columns(product_activity, monkeypatch): from activity_browser.bwutils.multilca import Contributions @@ -348,4 +417,21 @@ def test_top_ef_contributions_compare_impact_categories(lcia_overview_project): contributions = Contributions(mlca) df = contributions.top_elementary_flow_contributions(functional_unit="0", limit=5) assert len(df) > 0 - assert len(mlca.methods) == len(df.select_dtypes(include="number").columns) + numeric = list(df.select_dtypes(include="number").columns) + assert numeric == list(range(len(mlca.methods))) + + class Switches: + indexes = type("I", (), {"func": 0, "method": 1, "scenario": 2})() + + def currentIndex(self): + return self.indexes.method + + tab = type("Tab", (), {"switches": Switches(), "parent": type("P", (), {"mlca": mlca})()})() + labelled = apply_contribution_column_labels(df, tab) + assert list(labelled.columns[-len(mlca.methods) :]) == list(mlca.method_labels.values()) + + process_df = contributions.top_process_contributions(functional_unit="0", limit=5) + process_labelled = apply_contribution_column_labels(process_df, tab) + assert list(process_labelled.columns[-len(mlca.methods) :]) == list( + mlca.method_labels.values() + ) diff --git a/tests/test_sdf_comment_columns.py b/tests/test_sdf_comment_columns.py index 65aa37077..66559c810 100644 --- a/tests/test_sdf_comment_columns.py +++ b/tests/test_sdf_comment_columns.py @@ -1,11 +1,48 @@ -"""SDF comments: '#' rows (pandas comment=) and '_' columns (usecols). +"""SDF comments and required flow-scenario headers.""" +from openpyxl import Workbook -Keep this file free of Excel/openpyxl I/O so it stays cheap in CI. -Excel uses the same pandas knobs; column rule is covered by ``valid_cols``. -""" -from activity_browser.bwutils.superstructure.excel import valid_cols +from activity_browser.bwutils.superstructure.excel import ( + import_from_excel, + valid_cols, +) from activity_browser.bwutils.superstructure.file_imports import ABCSVImporter -from activity_browser.bwutils.superstructure.utils import SUPERSTRUCTURE +from activity_browser.bwutils.superstructure.utils import ( + SUPERSTRUCTURE, + is_flow_sdf_headers, + is_partial_flow_sdf_headers, + missing_superstructure_columns, +) + + +def _sample_row(amount: float = 1.0) -> list: + base = { + "from activity name": "A", + "from reference product": "p", + "from location": "GLO", + "from categories": "", + "from database": "db1", + "from key": "('db1', 'a')", + "to activity name": "B", + "to reference product": "q", + "to location": "GLO", + "to categories": "", + "to database": "db2", + "to key": "('db2', 'b')", + "flow type": "technosphere", + } + return [base[c] for c in SUPERSTRUCTURE] + ["note text", amount] + + +def _write_xlsx(path, rows) -> None: + wb = Workbook() + info = wb.active + info.title = "info" + info["A1"] = "info" + sheet = wb.create_sheet("scenarios") + for r_i, row in enumerate(rows, start=1): + for c_i, val in enumerate(row, start=1): + sheet.cell(r_i, c_i, val) + wb.save(path) def test_valid_cols_drops_underscore_prefix(): @@ -14,6 +51,14 @@ def test_valid_cols_drops_underscore_prefix(): assert valid_cols("from database") is True +def test_missing_superstructure_columns_detects_typo(): + cols = [c if c != "to key" else "to keyhhh" for c in SUPERSTRUCTURE] + ["2025"] + assert missing_superstructure_columns(cols) == ["to key"] + assert is_flow_sdf_headers(cols) is False + assert is_partial_flow_sdf_headers(cols) is True + assert is_flow_sdf_headers(list(SUPERSTRUCTURE) + ["2025"]) is True + + def test_csv_hash_rows_and_underscore_columns(tmp_path): cols = list(SUPERSTRUCTURE) + ["_notes", "2025"] header = ";".join(cols) @@ -38,3 +83,66 @@ def test_csv_hash_rows_and_underscore_columns(tmp_path): assert len(df) == 1 assert df.loc[0, "from database"] == "db1" assert float(df.loc[0, "2025"]) == 1.0 + + +def test_csv_misspelled_header_is_partial_sdf(tmp_path): + cols = [c if c != "to key" else "to keyhhh" for c in SUPERSTRUCTURE] + ["2025"] + header = ";".join(cols) + data = "A;p;GLO;;db1;('db1', 'a');B;q;GLO;;db2;('db2', 'b');technosphere;1.0" + path = tmp_path / "bad_header.csv" + path.write_text(header + "\n" + data + "\n", encoding="utf-8") + + df = ABCSVImporter.read_file(path, separator=";") + assert is_partial_flow_sdf_headers(df.columns) + assert missing_superstructure_columns(df.columns) == ["to key"] + + +def test_excel_leading_and_mid_hash_rows_with_underscore_column(tmp_path): + """Leading/mid '#' rows and '_' columns are excluded via skiprows/usecols.""" + cols = list(SUPERSTRUCTURE) + ["_notes", "2025"] + rows = [ + ["# leading file comment"] + [None] * (len(cols) - 1), + cols, + _sample_row(1.0), + ["# skipped data row"] + [None] * (len(cols) - 1), + _sample_row(2.0), + ] + path = tmp_path / "sdf.xlsx" + _write_xlsx(path, rows) + + df = import_from_excel(path, import_sheet=1) + + assert not df.empty + assert "from activity name" in df.columns + assert "_notes" not in df.columns + assert list(map(str, df.columns[-1:])) == ["2025"] + assert len(df) == 2 + assert float(df.iloc[0]["2025"]) == 1.0 + assert float(df.iloc[1]["2025"]) == 2.0 + + +def test_excel_misspelled_header_returns_partial_frame(tmp_path): + cols = [c if c != "to key" else "to keyhhh" for c in SUPERSTRUCTURE] + ["2025"] + path = tmp_path / "bad_header.xlsx" + _write_xlsx(path, [cols, _sample_row(1.0)]) + + df = import_from_excel(path, import_sheet=1) + + assert not df.empty + assert is_partial_flow_sdf_headers(df.columns) + assert missing_superstructure_columns(df.columns) == ["to key"] + assert "to keyhhh" in df.columns + + +def test_excel_underscore_column_first(tmp_path): + cols = ["_notes"] + list(SUPERSTRUCTURE) + ["2025"] + data = ["my note"] + _sample_row(3.0)[: len(SUPERSTRUCTURE)] + [3.0] + path = tmp_path / "sdf_first_notes.xlsx" + _write_xlsx(path, [cols, data]) + + df = import_from_excel(path, import_sheet=1) + + assert not df.empty + assert "_notes" not in df.columns + assert "from activity name" in df.columns + assert float(df.iloc[0]["2025"]) == 3.0 diff --git a/tests/test_substitution_sdf.py b/tests/test_substitution_sdf.py new file mode 100644 index 000000000..725119dcd --- /dev/null +++ b/tests/test_substitution_sdf.py @@ -0,0 +1,14 @@ +"""Scenario files may include flow type = substitution.""" + +from activity_browser.bwutils.superstructure.mlca import SuperstructureMLCA +from activity_browser.bwutils.utils import Index, Key + + +def test_substitution_scenario_rows_map_to_technosphere_matrix(): + assert SuperstructureMLCA.matrices["substitution"] == "technosphere_matrix" + idx = Index( + input=Key("db", "from"), + output=Key("db", "to"), + flow_type="substitution", + ) + assert idx.flip is False diff --git a/tests/test_webengine_wayland_flags.py b/tests/test_webengine_wayland_flags.py new file mode 100644 index 000000000..825742222 --- /dev/null +++ b/tests/test_webengine_wayland_flags.py @@ -0,0 +1,67 @@ +"""Pure-logic tests for Linux/Wayland Qt WebEngine GPU fallback flags.""" +import os + +import pytest + +from activity_browser.ui.core.application import ( + _WAYLAND_FLAGS, + _is_linux_wayland, + _webengine_flags, +) + + +@pytest.fixture(autouse=True) +def _clean_qt_env(monkeypatch): + for key in ( + "QT_QPA_PLATFORM", + "QTWEBENGINE_CHROMIUM_FLAGS", + "XDG_SESSION_TYPE", + "WAYLAND_DISPLAY", + ): + monkeypatch.delenv(key, raising=False) + + +def test_is_linux_wayland_by_session_type(monkeypatch): + monkeypatch.setattr("activity_browser.ui.core.application.sys.platform", "linux") + monkeypatch.setenv("XDG_SESSION_TYPE", "wayland") + assert _is_linux_wayland() is True + + +def test_is_linux_wayland_by_wayland_display(monkeypatch): + monkeypatch.setattr("activity_browser.ui.core.application.sys.platform", "linux") + monkeypatch.setenv("WAYLAND_DISPLAY", "wayland-0") + assert _is_linux_wayland() is True + + +def test_is_linux_wayland_false_on_xcb_override(monkeypatch): + monkeypatch.setattr("activity_browser.ui.core.application.sys.platform", "linux") + monkeypatch.setenv("XDG_SESSION_TYPE", "wayland") + monkeypatch.setenv("QT_QPA_PLATFORM", "xcb") + assert _is_linux_wayland() is False + + +def test_is_linux_wayland_false_on_windows(monkeypatch): + monkeypatch.setattr("activity_browser.ui.core.application.sys.platform", "win32") + monkeypatch.setenv("XDG_SESSION_TYPE", "wayland") + monkeypatch.setenv("WAYLAND_DISPLAY", "wayland-0") + assert _is_linux_wayland() is False + + +def test_webengine_flags_adds_wayland_gpu_disable(monkeypatch): + monkeypatch.setattr("activity_browser.ui.core.application.sys.platform", "linux") + monkeypatch.setenv("XDG_SESSION_TYPE", "wayland") + _webengine_flags() + flags = os.environ["QTWEBENGINE_CHROMIUM_FLAGS"].split() + for flag in _WAYLAND_FLAGS: + assert flag in flags + + +def test_webengine_flags_skips_wayland_on_xcb(monkeypatch): + monkeypatch.setattr("activity_browser.ui.core.application.sys.platform", "linux") + monkeypatch.setenv("XDG_SESSION_TYPE", "wayland") + monkeypatch.setenv("QT_QPA_PLATFORM", "xcb") + _webengine_flags() + assert "QTWEBENGINE_CHROMIUM_FLAGS" not in os.environ or not any( + f in os.environ.get("QTWEBENGINE_CHROMIUM_FLAGS", "").split() + for f in _WAYLAND_FLAGS + )