refactor(bazel): simplify dev/bazel: dead code, shared toolchain config, stock actions instead of helper scripts, docs

.bazelrc

@@ -1,10 +1,8 @@
 # Global options
 
-common  
-
 build -c opt \
       --incompatible_enable_cc_toolchain_resolution \
-      --incompatible_require_linker_input_cc_api 
+      --incompatible_require_linker_input_cc_api
 
 # Aliases for user-defined flags
 build --flag_alias=backend_config=@config//:backend_config

MODULE.bazel

@@ -32,12 +32,6 @@ sh_config_ext = use_extension("@onedal//dev/bazel/toolchains:cc_toolchain_extens
 use_repo(sh_config_ext, "onedal_cc_toolchain")
 register_toolchains("@{}//:all".format("onedal_cc_toolchain"))
 
-
-extra_toolchain_ext = use_extension("@onedal//dev/bazel/toolchains:extra_toolchain_extension.bzl", "onedal_extra_toolchain_extension")
-use_repo(extra_toolchain_ext, "onedal_extra_toolchain")
-register_toolchains("@{}//:all".format("onedal_extra_toolchain"))
-
-
 http_archive = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 http_archive(
     name = "catch2",

dev/bazel/README.md

@@ -16,7 +16,8 @@
 *******************************************************************************/-->
 # Bazel Guide
 ## Install Bazel on Linux
-1. Download Bazelisk
+1. Download Bazelisk. `.ci/env/bazelisk.sh` does the same with a pinned
+   version and a checksum check, and is what CI runs.
    ```sh
    wget -O bazel https://github.com/bazelbuild/bazelisk/releases/download/v1.29.0/bazelisk-linux-amd64
    ```
@@ -52,9 +53,10 @@ validation.
 1. Install Visual Studio 2022 Build Tools with the MSVC x64 C++ toolchain.
 
 2. Download Bazelisk for Windows and put it into a directory on `PATH`, or keep
-   it in the repository root as `bazelisk.exe`.
+   it in the repository root as `bazelisk.exe`. `.ci/env/bazelisk.ps1` does this
+   with a pinned version and a checksum check, and is what CI runs.
    ```bat
-   set BAZELISK_VERSION=v1.28.1
+   set BAZELISK_VERSION=v1.29.0
    curl.exe -L -o bazelisk.exe https://github.com/bazelbuild/bazelisk/releases/download/%BAZELISK_VERSION%/bazelisk-windows-amd64.exe
    bazelisk.exe version
    ```
@@ -225,18 +227,19 @@ The most used Bazel commands are `build`, `test` and `run`.
 
 - `--cpu` CPU instruction sets to compile library for. \
   Possible values:
-  - `auto` _(default)_ Automatically detects highest available instruction set
-    on the local machine. If detection failed, uses `avx2`.
+  - `auto` _(default)_ Automatically detects the highest instruction set
+    available on the local machine and builds it together with the `sse2`
+    dispatch baseline, which is always included on x86.
   - `modern` Compiles for `sse2`, `avx2`, `avx512`.
   - `all` Compiles for all instruction sets listed below.
-  - Any comma-separated combination of the following values:
+  - Any space-separated combination of the following values:
     - `sse2`
     - `avx2`
     - `avx512`
 
    Example:
    ```sh
-   bazel test --cpu="avx2,avx512" //cpp/oneapi/dal:tests
+   bazel test --cpu="avx2 avx512" //cpp/oneapi/dal:tests
    ```
 
 - `--test_external_datasets` A switch that enables
@@ -407,7 +410,10 @@ driver in DPC++ link actions while moving large object lists to a response
 file. Each script contains detailed usage and maintenance comments.
 
 When package contents change, update the Bazel packaging rules or the common
-comparator rather than hiding differences in a platform wrapper.
+comparator rather than hiding differences in a platform wrapper. The same goes
+for the build itself: prefer a Bazel action to a helper script. The scripts that
+remain are listed in `dev/bazel/TODO.md` together with what each of them is
+waiting on.
 
 ### Run oneAPI examples
 - To run all oneAPI C++ example use the following commands:
@@ -561,9 +567,6 @@ dal_test_suite(
      bazel test --test_link_mode=release_dynamic //cpp/oneapi/dal:tests
      ```
 
-## What is missing in this guide
-- How to get make-like release structure
-
 ## Standard-library allocator
 
 For Linux DAAL core objects, this is equivalent to the allocator-selection

dev/bazel/TODO.md

@@ -15,22 +15,71 @@
 * limitations under the License.
 *******************************************************************************/-->
 
-### TODO:
-- [ ] **Windows support.** Bazel shall provide toolchain implementation for
-  Windows.
+### Done:
+
+- [x] **Windows support.** `toolchains/cc_toolchain_win.bzl` configures Intel
+  `icx` and, when it is not on `PATH`, MSVC `cl` through rules_cc's
+  auto-configuration. `//:release` and `//:release_all` build on Windows in both
+  CRT flavours and are covered by CI. Running DPC++ examples and tests there is
+  still open, see below.
 
-- [ ] **Release to oneAPI structure.** Bazel shall write headers, binaries,
-  examples and scripts to oneAPI release structure as the current make does.
+- [x] **Release to oneAPI structure.** `//:release` writes headers, libraries,
+  examples, environment scripts, pkg-config and CMake metadata to
+  `bazel-bin/release/daal/latest`. Nightly CI compares that tree against the
+  Make one with `dev/release_tests/compare_release_trees.py`.
+
+- [x] **Automatic host architecture identification.** `--cpu=auto`, the default,
+  detects the highest instruction set available on the host and adds the `sse2`
+  baseline.
+
+### TODO:
 
 - [ ] **Extend compiler support matrix.** Current status:
   |         |        Intel       |        DPC++       |         GCC        |       Clang        |        MSVC        |
   |---------|:------------------:|:------------------:|:------------------:|:------------------:|:------------------:|
-  | Linux   |        :x:         |         :x:        | :heavy_check_mark: |                    |                    |
-  | Windows |        :x:         |         :x:        |                    |                    |        :x:         |
+  | Linux   | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |                    |
+  | Windows | :heavy_check_mark: | :heavy_check_mark: |                    |        :x:         | :heavy_check_mark: |
+
+  Intel `icx`/`icpx` is preferred whenever it is on `PATH`; `CC` overrides the
+  choice. Linux Clang builds `//:release`, but no CI job selects it, so
+  regressions in that flag set are only found by hand. Windows Clang
+  (`clang-cl`) is added by #3795.
+
+- [ ] **Windows DPC++ execution.** Windows release artifacts can include the
+  DPC++ libraries, but the examples and tests that create a SYCL queue are not
+  run there yet; that needs device and runtime validation on the CI images.
 
-- [ ] **Automatic host architecture identification.** Bazel shall detect host
-  machine architecture and configure best CPU id automatically.
+- [ ] **Toolchain flag tables as data.** The Linux and Windows
+  `cc_toolchain_config` rules now share their action groups, their
+  attribute-driven features and their rule attributes through
+  `toolchains/cc_toolchain_config_common.bzl`. What is left in the two files is
+  not duplication: no same-named feature emits the same flags on both
+  platforms, so folding them together would put a per-platform branch inside
+  every factory. The duplication that is real sits in `toolchains/common.bzl`,
+  where the per-compiler, per-OS, per-ISA flag lists are spelled out in
+  Starlark control flow and would read better as a table.
 
-- [ ] **Toolchain code unification.** There is logic duplication for toolchain
-  configuration on Linux/Windows.
+- [ ] **Hermetic toolchain.** The compiler, archiver, linker and strip tool are
+  absolute paths found with `repo_ctx.which()` at configure time. A build
+  therefore depends on the host `PATH`, and nothing invalidates the action cache
+  when the compiler behind that path changes. Registering a downloaded toolchain
+  would fix both.
 
+- [ ] **Remove the remaining helper scripts.** Four are left, and each waits on
+  something Bazel or rules_cc does not expose:
+  - `toolchains/tools/dll_to_implib.bat` — `cc_common.link()` does not register
+    `supports_interface_shared_libraries` for this toolchain config and cannot
+    declare `-IMPLIB:`'s side-effect file as an output, so the import library is
+    derived from `dumpbin` output after the link.
+  - `toolchains/tools/copy_crlf.bat` — the release tree needs CRLF text files
+    and no Bazel action rewrites line endings.
+  - `toolchains/tools/dpc_link_win.ps1` and its `.tpl.bat` launcher — keeps the
+    Intel driver in DPC++ link actions while spilling long object lists into a
+    response file, which the driver needs and Bazel's own param-file support
+    does not provide for a wrapped tool.
+  - `toolchains/tools/tool_not_found.tpl.sh` and `.tpl.bat` — stand in for an
+    optional tool that is not installed, today only the DPC++ compiler, and
+    fail the action that reaches them. The failure belongs in analysis instead:
+    `@config` already knows whether DPC++ was found, so a DPC++ target could
+    `select()` onto an error target the way `daal_module` does for `--stdalloc`
+    on non-Linux.

dev/bazel/cc.bzl

@@ -15,7 +15,6 @@
 #===============================================================================
 
 load("@onedal//dev/bazel:utils.bzl",
-    "utils",
     "paths",
     "sets",
 )
@@ -317,31 +316,39 @@ cc_static_lib = rule(
 )
 
 
-def _copy_dynamic_release_file(ctx, src, out_name, is_windows = False, extra_inputs = []):
+def _copy_windows_release_file(ctx, src, out_name, extra_inputs = []):
+    """Copy a linker output under the file name the release layout expects.
+
+    Windows only: the Linux release names come straight out of the link action,
+    while a DLL and its import library have to be renamed afterwards (see
+    `_cc_dynamic_lib_impl`). A symlink is not enough, because creating one
+    requires developer mode on Windows.
+
+    Args:
+        ctx: rule context.
+        src: the file produced by the link action.
+        out_name: base name of the copy, declared in the current package.
+        extra_inputs: further link outputs to declare as inputs, so that the
+                      copy cannot run before the whole link action has completed.
+
+    Returns:
+        The declared copy.
+    """
     out = ctx.actions.declare_file(out_name)
-    if is_windows:
-        ctx.actions.run(
-            executable = "cmd.exe",
-            inputs = [src] + extra_inputs,
-            outputs = [out],
-            arguments = [
-                "/d",
-                "/c",
-                'copy /Y "{}" "{}"'.format(
-                    src.path.replace("/", "\\"),
-                    out.path.replace("/", "\\"),
-                ),
-            ],
-            use_default_shell_env = True,
-        )
-    else:
-        ctx.actions.run(
-            executable = "cp",
-            inputs = [src] + extra_inputs,
-            outputs = [out],
-            arguments = [src.path, out.path],
-            use_default_shell_env = True,
-        )
+    ctx.actions.run(
+        executable = "cmd.exe",
+        inputs = [src] + extra_inputs,
+        outputs = [out],
+        arguments = [
+            "/d",
+            "/c",
+            'copy /Y "{}" "{}"'.format(
+                src.path.replace("/", "\\"),
+                out.path.replace("/", "\\"),
+            ),
+        ],
+        use_default_shell_env = True,
+    )
     return out
 
 
@@ -407,19 +414,17 @@ def _cc_dynamic_lib_impl(ctx):
             if dynamic_outputs.dynamic_library.basename == dynamic_release_name:
                 default_files.append(dynamic_outputs.dynamic_library)
             else:
-                default_files.append(_copy_dynamic_release_file(
+                default_files.append(_copy_windows_release_file(
                     ctx,
                     dynamic_outputs.dynamic_library,
                     dynamic_release_name,
-                    is_windows = is_windows,
                     extra_inputs = [dynamic_outputs.interface_library] if dynamic_outputs.interface_library else [],
                 ))
         if dynamic_outputs.interface_library:
-            default_files.append(_copy_dynamic_release_file(
+            default_files.append(_copy_windows_release_file(
                 ctx,
                 dynamic_outputs.interface_library,
                 "{}_dll.lib".format(rt_name),
-                is_windows = is_windows,
             ))
     default_info = DefaultInfo(
         files = depset(default_files),

dev/bazel/cc/common.bzl

@@ -19,7 +19,6 @@ load("@rules_cc//cc/common:cc_info.bzl", "CcInfo")
 
 load("@onedal//dev/bazel:utils.bzl",
     "utils",
-    "paths",
     "sets",
 )