Add workspace Key Vault secrets retrieval API for workspace services and user resources

CHANGELOG.md

@@ -6,6 +6,8 @@
 * Add data science tooling (Azure CLI, VS Code, Storage Explorer, Git, Python/JupyterLab, R/RStudio) to Guacamole Windows VMs via a shared `vm_config.ps1` bootstrap pulled through Nexus. Existing `tre-service-guacamole-windowsvm`, `tre-service-guacamole-import-reviewvm`, and `tre-service-guacamole-export-reviewvm` resources **must not be upgraded** to these new versions — redeploy instead. Upgrade the Nexus shared service to `sonatype-nexus` 3.9.0 before deploying the new Windows VM templates to ensure the required proxy repositories are available. (`tre-service-guacamole-windowsvm` 3.0.0, `tre-service-guacamole-import-reviewvm`/`tre-service-guacamole-export-reviewvm` 2.0.0, `sonatype-nexus` 3.9.0) ([#4981](https://github.com/microsoft/AzureTRE/pull/4981))
 
 ENHANCEMENTS:
+* Add secrets retrieval API endpoints for workspace services and user resources so researchers can access secrets (e.g. storage keys, database connection details) that resources store in the workspace Key Vault. Resource properties whose name contains `keyvault_secret_id` are treated as references to workspace Key Vault secrets, and the UI adds a reveal/copy affordance to display these secret values on demand. Key Vault is accessed through an On-Behalf-Of exchange authenticated by the per-workspace app registration via a federated identity credential (no stored client secret), so secrets are read as the signed-in user rather than the core API identity. (API 0.28.0, `tre-workspace-base` 2.12.0, UI 0.9.0) ([#2402](https://github.com/microsoft/AzureTRE/issues/2402))
+* Grant the owning user `Key Vault Secrets User` (secrets reader) scoped to their own VM admin password secret on the Guacamole Windows and Linux VMs, so users get least-privilege read access to their own credentials. Airlock review VMs are unaffected as they use ephemeral random credentials. (`tre-service-guacamole-windowsvm` 3.1.0, `tre-service-guacamole-linuxvm` 1.5.0, `tre-service-guacamole-import-reviewvm`/`tre-service-guacamole-export-reviewvm` 2.0.1) ([#2402](https://github.com/microsoft/AzureTRE/issues/2402))
 * Enable graceful upgrading of the Nexus shared service: modified or added repository configs and the container image (now `3.94.0`) are applied to the existing VM on upgrade without recreating it. Removed the non-functional `snapcraft` proxy (dead remote URL) and skip repositories left in a failed state so they don't block upgrades. (`sonatype-nexus` 3.10.0) ([#2721](https://github.com/microsoft/AzureTRE/issues/2721))
 * Specify default_outbound_access_enabled = false setting for all subnets ([#4757](https://github.com/microsoft/AzureTRE/pull/4757))
 * Pin all GitHub Actions workflow steps to full commit SHAs to prevent supply chain attacks plus update to latest releases ([#4886](https://github.com/microsoft/AzureTRE/pull/4886))

api_app/_version.py

@@ -1 +1 @@
-__version__ = "0.26.0"
+__version__ = "0.28.0"

api_app/api/routes/workspaces.py

@@ -22,6 +22,7 @@
 from models.schemas.workspace_service import WorkspaceServiceInCreate, WorkspaceServicesInList, WorkspaceServiceInResponse
 from models.schemas.resource import ResourceHistoryInList, ResourcePatch
 from models.schemas.resource_template import ResourceTemplateInformationInList
+from models.schemas.secret import SecretInResponse
 from resources import strings
 from services.aad_authentication import AuthConfigValidationError
 from auth.rbac import require_tre_admin, require_workspace_owner, \
@@ -30,6 +31,7 @@
     require_workspace_owner_or_researcher_or_airlock_manager_or_tre_admin
 from services.authentication import get_aad_service, extract_auth_information
 from services.azure_resource_status import get_azure_resource_status
+from services.secrets import get_secret_value, is_secret_property, SecretRetrievalError
 from azure.cosmos.exceptions import CosmosAccessConditionFailedError
 
 from .resource_helpers import cascaded_update_resource, delete_validation, enrich_resource_with_available_upgrades, get_identity_role_assignments, save_and_deploy_resource, construct_location_header, send_uninstall_message, \
@@ -53,6 +55,33 @@ def validate_user_has_valid_role_for_user_resource(user, user_resource):
     raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=strings.ACCESS_USER_IS_NOT_OWNER_OR_RESEARCHER)
 
 
+def _extract_bearer_token(authorization: str) -> str:
+    """Return the raw bearer token from an Authorization header value, or None."""
+    if authorization and authorization.lower().startswith("bearer "):
+        return authorization[len("bearer "):].strip()
+    return None
+
+
+async def retrieve_resource_secret(resource, secret_name: str, workspace, user_token: str) -> SecretInResponse:
+    if not is_secret_property(secret_name):
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=strings.SECRET_PROPERTY_IS_NOT_A_SECRET)
+
+    keyvault_secret_id = resource.properties.get(secret_name)
+    if not keyvault_secret_id:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=strings.SECRET_PROPERTY_DOES_NOT_EXIST)
+
+    try:
+        secret_value = await get_secret_value(
+            keyvault_secret_id,
+            workspace.properties.get("keyvault_uri"),
+            workspace.properties.get("client_id"),
+            user_token)
+    except SecretRetrievalError as e:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(e))
+
+    return SecretInResponse(key=secret_name, value=secret_value)
+
+
 # WORKSPACE ROUTES
 @workspaces_core_router.get("/workspaces", response_model=WorkspacesInList, name=strings.API_GET_ALL_WORKSPACES)
 async def retrieve_users_active_workspaces(user=Depends(require_tre_user_or_admin), workspace_repo=Depends(get_repository(WorkspaceRepository)), resource_template_repo=Depends(get_repository(ResourceTemplateRepository))) -> WorkspacesInList:
@@ -241,6 +270,11 @@ async def retrieve_workspace_service_by_id(workspace_service=Depends(get_workspa
     return WorkspaceServiceInResponse(workspaceService=workspace_service)
 
 
+@workspace_services_workspace_router.get("/workspaces/{workspace_id}/workspace-services/{service_id}/secrets/{secret_name}", response_model=SecretInResponse, name=strings.API_GET_WORKSPACE_SERVICE_SECRET, dependencies=[Depends(require_workspace_owner_or_researcher_or_airlock_manager)])
+async def retrieve_workspace_service_secret(secret_name: str, workspace=Depends(get_workspace_by_id_from_path), workspace_service=Depends(get_workspace_service_by_id_from_path), authorization: str = Header(None)) -> SecretInResponse:
+    return await retrieve_resource_secret(workspace_service, secret_name, workspace, _extract_bearer_token(authorization))
+
+
 @workspace_services_workspace_router.post("/workspaces/{workspace_id}/workspace-services", status_code=status.HTTP_202_ACCEPTED, response_model=OperationInResponse, name=strings.API_CREATE_WORKSPACE_SERVICE, dependencies=[Depends(require_workspace_owner)])
 async def create_workspace_service(response: Response, workspace_service_input: WorkspaceServiceInCreate, user=Depends(require_workspace_owner), workspace_service_repo=Depends(get_repository(WorkspaceServiceRepository)), workspace_repo=Depends(get_repository(WorkspaceRepository)), resource_template_repo=Depends(get_repository(ResourceTemplateRepository)), operations_repo=Depends(get_repository(OperationRepository)), resource_history_repo=Depends(get_repository(ResourceHistoryRepository)), workspace=Depends(get_deployed_workspace_by_id_from_path)) -> OperationInResponse:
 
@@ -400,6 +434,17 @@ async def retrieve_user_resource_by_id(
     return UserResourceInResponse(userResource=user_resource)
 
 
+@user_resources_workspace_router.get("/workspaces/{workspace_id}/workspace-services/{service_id}/user-resources/{resource_id}/secrets/{secret_name}", response_model=SecretInResponse, name=strings.API_GET_USER_RESOURCE_SECRET, dependencies=[Depends(get_workspace_by_id_from_path)])
+async def retrieve_user_resource_secret(
+        secret_name: str,
+        workspace=Depends(get_workspace_by_id_from_path),
+        user_resource=Depends(get_user_resource_by_id_from_path),
+        authorization: str = Header(None),
+        user=Depends(require_workspace_owner_or_researcher_or_airlock_manager)) -> SecretInResponse:
+    validate_user_has_valid_role_for_user_resource(user, user_resource)
+    return await retrieve_resource_secret(user_resource, secret_name, workspace, _extract_bearer_token(authorization))
+
+
 @user_resources_workspace_router.post("/workspaces/{workspace_id}/workspace-services/{service_id}/user-resources", status_code=status.HTTP_202_ACCEPTED, response_model=OperationInResponse, name=strings.API_CREATE_USER_RESOURCE)
 async def create_user_resource(
         response: Response,

api_app/models/schemas/secret.py

@@ -0,0 +1,14 @@
+from pydantic import BaseModel, Field
+
+
+class SecretInResponse(BaseModel):
+    key: str = Field("", title="Property name", description="Name of the resource property that references the secret")
+    value: str = Field("", title="Secret value", description="The secret value retrieved from the workspace Key Vault")
+
+    class Config:
+        schema_extra = {
+            "example": {
+                "key": "admin_password_keyvault_secret_id",
+                "value": "a-very-secret-value"
+            }
+        }

api_app/requirements.txt

@@ -3,6 +3,7 @@ azure-core==1.38.0
 azure-cosmos==4.14.3
 azure-eventgrid==4.22.0
 azure-identity==1.25.1
+azure-keyvault-secrets==4.11.0
 azure-mgmt-compute==37.1.0
 azure-mgmt-cosmosdb==9.9.0
 azure-mgmt-costmanagement==4.0.1

api_app/resources/strings.py

@@ -23,6 +23,7 @@
 
 API_GET_ALL_WORKSPACE_SERVICES = "Get all workspace services for workspace"
 API_GET_WORKSPACE_SERVICE_BY_ID = "Get workspace service by Id"
+API_GET_WORKSPACE_SERVICE_SECRET = "Get a workspace service secret value from the workspace Key Vault"
 API_CREATE_WORKSPACE_SERVICE = "Create a workspace service"
 API_UPDATE_WORKSPACE_SERVICE = "Update an existing workspace service"
 API_DELETE_WORKSPACE_SERVICE = "Delete workspace service"
@@ -34,6 +35,7 @@
 API_CREATE_USER_RESOURCE = "Create a user resource"
 API_GET_MY_USER_RESOURCES = "Get my user resources in the workspace service"
 API_GET_USER_RESOURCE = "Get user resource by id"
+API_GET_USER_RESOURCE_SECRET = "Get a user resource secret value from the workspace Key Vault"
 API_DELETE_USER_RESOURCE = "Delete user resource"
 API_UPDATE_USER_RESOURCE = "Update an existing user resource"
 API_INVOKE_ACTION_ON_USER_RESOURCE = "Invoke action on a user resource"
@@ -148,6 +150,13 @@
 CUSTOM_ACTION_NOT_DEFINED = "The specified custom action isn't defined in the targeted resource."
 CUSTOM_ACTIONS_DO_NOT_EXIST = "The resource being targeted does not implement any custom actions."
 
+SECRET_PROPERTY_DOES_NOT_EXIST = "The resource does not have a property with the requested name"
+SECRET_PROPERTY_IS_NOT_A_SECRET = "The requested property does not reference a Key Vault secret"
+INVALID_KEYVAULT_SECRET_ID = "The property value is not a valid Key Vault secret identifier"
+KEYVAULT_SECRET_OUTSIDE_WORKSPACE = "The referenced secret does not belong to the workspace Key Vault"
+KEYVAULT_SECRET_NOT_FOUND = "The secret could not be found in the workspace Key Vault"
+UNABLE_TO_RETRIEVE_KEYVAULT_SECRET = "Unable to retrieve the secret from the workspace Key Vault"
+
 WORKSPACE_SERVICE_TEMPLATE_DOES_NOT_EXIST = "Could not retrieve the workspace service template specified"
 TEMPLATE_DOES_NOT_EXIST = "Could not retrieve the template with this name, or name-version pair"
 NO_UNIQUE_CURRENT_FOR_TEMPLATE = "The template has multiple 'current' versions"

api_app/services/secrets.py

@@ -0,0 +1,110 @@
+from typing import Optional
+from urllib.parse import urlparse
+
+from azure.core.exceptions import ResourceNotFoundError
+from azure.identity import ManagedIdentityCredential
+from azure.identity.aio import OnBehalfOfCredential
+from azure.keyvault.secrets import KeyVaultSecretIdentifier
+from azure.keyvault.secrets.aio import SecretClient
+
+from core.config import AAD_AUTHORITY_URL, AAD_TENANT_ID, MANAGED_IDENTITY_CLIENT_ID
+from resources import strings
+from services.logging import logger
+
+
+# Convention: any resource property whose name contains this token holds a
+# Key Vault secret identifier rather than the secret value itself.
+KEYVAULT_SECRET_ID_TOKEN = "keyvault_secret_id"
+
+# Audience used when the API's managed identity requests a token to use as the
+# client assertion in the On-Behalf-Of exchange. The managed identity is
+# configured as a federated identity credential on the workspace app
+# registration, so this token replaces a stored client secret.
+TOKEN_EXCHANGE_SCOPE = "api://AzureADTokenExchange/.default"
+
+
+def is_secret_property(property_name: str) -> bool:
+    """Return True if the property name follows the secret naming convention."""
+    return KEYVAULT_SECRET_ID_TOKEN in property_name
+
+
+def _get_client_assertion() -> str:
+    """Return a managed identity token to use as the OBO client assertion.
+
+    The API's managed identity is registered as a federated identity credential
+    on the workspace app registration. A managed identity token for the token
+    exchange audience is therefore accepted by Entra ID in place of a workspace
+    client secret, so no secret needs to be stored or rotated.
+    """
+    credential = ManagedIdentityCredential(client_id=MANAGED_IDENTITY_CLIENT_ID)
+    try:
+        return credential.get_token(TOKEN_EXCHANGE_SCOPE).token
+    finally:
+        credential.close()
+
+
+def _get_obo_credential(workspace_client_id: str, user_token: str) -> OnBehalfOfCredential:
+    """Build an On-Behalf-Of credential for the workspace app registration.
+
+    The workspace app registration acts as the confidential client, using the
+    federated managed identity assertion to authenticate, and exchanges the
+    caller's token so that downstream Key Vault access is performed on behalf of
+    the signed-in user.
+    """
+    return OnBehalfOfCredential(
+        tenant_id=AAD_TENANT_ID,
+        client_id=workspace_client_id,
+        user_assertion=user_token,
+        client_assertion_func=_get_client_assertion,
+        authority=urlparse(AAD_AUTHORITY_URL).netloc,
+    )
+
+
+async def get_secret_value(
+    keyvault_secret_id: str,
+    expected_keyvault_uri: Optional[str] = None,
+    workspace_client_id: Optional[str] = None,
+    user_token: Optional[str] = None,
+) -> str:
+    """Retrieve a secret value from a workspace Key Vault on behalf of the user.
+
+    The ``keyvault_secret_id`` is a full Key Vault secret identifier (URI) as
+    output by a resource template. When ``expected_keyvault_uri`` is provided the
+    secret's vault must match it, ensuring a resource can only expose secrets that
+    live in its own workspace Key Vault.
+
+    ``workspace_client_id`` (the workspace app registration client id) and
+    ``user_token`` (the caller's access token) are used to perform an
+    On-Behalf-Of exchange so Key Vault access is authorised as the signed-in
+    user rather than the API's own identity.
+    """
+    if not workspace_client_id or not user_token:
+        raise SecretRetrievalError(strings.UNABLE_TO_RETRIEVE_KEYVAULT_SECRET)
+
+    try:
+        parsed_secret_id = KeyVaultSecretIdentifier(keyvault_secret_id)
+    except ValueError:
+        raise SecretRetrievalError(strings.INVALID_KEYVAULT_SECRET_ID)
+
+    if expected_keyvault_uri and not _same_vault(parsed_secret_id.vault_url, expected_keyvault_uri):
+        raise SecretRetrievalError(strings.KEYVAULT_SECRET_OUTSIDE_WORKSPACE)
+
+    async with _get_obo_credential(workspace_client_id, user_token) as credential:
+        async with SecretClient(vault_url=parsed_secret_id.vault_url, credential=credential) as secret_client:
+            try:
+                secret = await secret_client.get_secret(parsed_secret_id.name)
+            except ResourceNotFoundError:
+                raise SecretRetrievalError(strings.KEYVAULT_SECRET_NOT_FOUND)
+            except Exception:
+                logger.exception("Failed to retrieve secret from Key Vault")
+                raise SecretRetrievalError(strings.UNABLE_TO_RETRIEVE_KEYVAULT_SECRET)
+
+    return secret.value
+
+
+def _same_vault(vault_url_a: str, vault_url_b: str) -> bool:
+    return urlparse(vault_url_a).netloc.lower() == urlparse(vault_url_b).netloc.lower()
+
+
+class SecretRetrievalError(Exception):
+    """Raised when a workspace Key Vault secret cannot be retrieved."""