-
Notifications
You must be signed in to change notification settings - Fork 3
📦Adding acm-member-portal deployment #158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| --- | ||
| name: 'Terraform Apply on Kubernetes' | ||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| paths: | ||
| - '.github/workflows/k8s_terraform_apply.yml' | ||
| - 'kubernetes/terraform/stacks/acm-member-portal/**' | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
| id-token: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| terraform: | ||
| name: 'Terraform Apply' | ||
| env: | ||
| ARM_CLIENT_ID: ${{ secrets.AZURE_AD_CLIENT_ID }} | ||
| ARM_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }} | ||
| ARM_TENANT_ID: ${{ secrets.AZURE_AD_TENANT_ID }} | ||
| ARM_USE_OIDC: true | ||
| TF_WORKSPACE: prod | ||
| TF_IN_AUTOMATION: true | ||
| TF_INPUT: 0 | ||
| runs-on: maid-cafe | ||
| environment: prod | ||
|
|
||
| strategy: | ||
| matrix: | ||
| stack: | ||
| - acm-member-portal | ||
|
|
||
| defaults: | ||
| run: | ||
| shell: bash | ||
| working-directory: kubernetes/terraform/stacks/${{ matrix.stack }} | ||
|
|
||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Azure CLI setup | ||
| uses: azure/login@v1 | ||
| with: | ||
| client-id: ${{ env.ARM_CLIENT_ID }} | ||
| tenant-id: ${{ env.ARM_TENANT_ID }} | ||
| subscription-id: ${{ env.ARM_SUBSCRIPTION_ID }} | ||
|
|
||
| - name: 'Terraform Format' | ||
| id: fmt | ||
| uses: dflook/terraform-fmt-check@v1 | ||
| with: | ||
| path: "kubernetes/terraform/stacks/${{ matrix.stack }}" | ||
| workspace: ${{ env.TF_WORKSPACE }} | ||
|
|
||
| - name: 'Terraform Validate' | ||
| id: validate | ||
| uses: dflook/terraform-validate@v1 | ||
| with: | ||
| path: "kubernetes/terraform/stacks/${{ matrix.stack }}" | ||
| workspace: ${{ env.TF_WORKSPACE }} | ||
|
|
||
| - name: 'Terraform Apply' | ||
| id: apply | ||
| uses: dflook/terraform-apply@v1 | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| with: | ||
| path: "kubernetes/terraform/stacks/${{ matrix.stack }}" | ||
| workspace: ${{ env.TF_WORKSPACE }} | ||
| var_file: "kubernetes/terraform/stacks/${{ matrix.stack }}/configuration/${{ env.TF_WORKSPACE }}.tfvars" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| --- | ||
| name: 'Terraform PRs for Kubernetes' | ||
| on: | ||
| pull_request: | ||
| paths: | ||
| - '.github/workflows/k8s_terraform_plan.yml' | ||
| - 'kubernetes/terraform/stacks/acm-member-portal/**' | ||
|
|
||
| permissions: | ||
| contents: read | ||
| id-token: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| terraform: | ||
| name: 'Terraform PR Plan' | ||
| env: | ||
| ARM_CLIENT_ID: ${{ secrets.AZURE_AD_CLIENT_ID }} | ||
| ARM_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }} | ||
| ARM_TENANT_ID: ${{ secrets.AZURE_AD_TENANT_ID }} | ||
| ARM_USE_OIDC: true | ||
| TF_WORKSPACE: prod | ||
| TF_IN_AUTOMATION: true | ||
| TF_INPUT: 0 | ||
| runs-on: maid-cafe | ||
|
|
||
| strategy: | ||
| matrix: | ||
| stack: | ||
| - acm-member-portal | ||
|
|
||
| defaults: | ||
| run: | ||
| shell: bash | ||
| working-directory: kubernetes/terraform/stacks/${{ matrix.stack }} | ||
|
|
||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Azure CLI setup | ||
| uses: azure/login@v1 | ||
| with: | ||
| client-id: ${{ env.ARM_CLIENT_ID }} | ||
| tenant-id: ${{ env.ARM_TENANT_ID }} | ||
| subscription-id: ${{ env.ARM_SUBSCRIPTION_ID }} | ||
|
|
||
| - name: 'Terraform Format' | ||
| id: fmt | ||
| uses: dflook/terraform-fmt-check@v1 | ||
| with: | ||
| path: "kubernetes/terraform/stacks/${{ matrix.stack }}" | ||
| workspace: ${{ env.TF_WORKSPACE }} | ||
|
|
||
| - name: 'Terraform Validate' | ||
| id: validate | ||
| uses: dflook/terraform-validate@v1 | ||
| with: | ||
| path: "kubernetes/terraform/stacks/${{ matrix.stack }}" | ||
| workspace: ${{ env.TF_WORKSPACE }} | ||
|
|
||
| - name: 'Terraform Plan' | ||
| id: plan | ||
| uses: dflook/terraform-plan@v1 | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| with: | ||
| path: "kubernetes/terraform/stacks/${{ matrix.stack }}" | ||
| workspace: ${{ env.TF_WORKSPACE }} | ||
| var_file: "kubernetes/terraform/stacks/${{ matrix.stack }}/configuration/${{ env.TF_WORKSPACE }}.tfvars" |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # ACM member portal Terraform stack | ||
|
|
||
| Entra app registration for the member portal, plus the `portal-secrets` Kubernetes Secret (`BETTER_AUTH_SECRET` and `MICROSOFT_*`). | ||
|
|
||
| Helm / Argo CD deploy the app from `kubernetes/argocd/stacks/acm-member-portal`. SMTP, Discord, and Windows API keys are not managed here. Add them to the same Secret after apply. `kubernetes_secret_v1` replaces the whole object, so those extra keys are wiped on the next apply or client-secret rotation. Re-add them, or move them to a second Secret. | ||
|
|
||
| ## Requirements | ||
|
|
||
| - Azure login with permission to create app registrations (`az login`, or the CI OIDC service principal) | ||
| - `KUBE_CONFIG_PATH` or `KUBECONFIG` pointing at the cluster that hosts `acm-portal` | ||
| - Namespace `acm-portal` already present (Argo CD creates it) | ||
|
|
||
| ## Manual run | ||
|
|
||
| ```bash | ||
| terraform init | ||
| terraform workspace select prod || terraform workspace new prod | ||
| terraform plan -var-file configuration/prod.tfvars | ||
| terraform apply -var-file configuration/prod.tfvars | ||
| ``` | ||
|
|
||
| After apply, in Entra: App registrations → `acm-member-portal-prod` → API permissions → Grant admin consent. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| data "azuread_client_config" "current" {} | ||
|
|
||
| locals { | ||
| owners = distinct(concat([data.azuread_client_config.current.object_id], var.additional_owner_ids)) | ||
| } | ||
|
|
||
| resource "azuread_application" "portal" { | ||
| display_name = "acm-member-portal-${var.deployment_env}" | ||
| sign_in_audience = "AzureADMyOrg" | ||
| owners = local.owners | ||
|
|
||
| web { | ||
| redirect_uris = var.redirect_uris | ||
| logout_url = length(var.logout_uris) > 0 ? var.logout_uris[0] : null | ||
| } | ||
|
|
||
| # Graph delegated scopes. openid / profile / email are requested at runtime by | ||
| # better-auth; grant admin consent in Entra after the first apply. | ||
| required_resource_access { | ||
| resource_app_id = "00000003-0000-0000-c000-000000000000" # Microsoft Graph | ||
|
|
||
| resource_access { | ||
| id = "37f7f235-527c-4136-accd-4f02faf2b745" | ||
| type = "Scope" # openid | ||
| } | ||
| resource_access { | ||
| id = "64a6cdd6-4a97-4e60-a3e8-d57ef873a778" | ||
| type = "Scope" # email | ||
| } | ||
| } | ||
|
|
||
| optional_claims { | ||
| id_token { name = "email" } | ||
| id_token { name = "upn" } | ||
| id_token { | ||
| name = "groups" | ||
| additional_properties = ["sam_account_name"] | ||
| } | ||
| } | ||
| } | ||
|
|
||
| resource "azuread_service_principal" "portal" { | ||
| client_id = azuread_application.portal.client_id | ||
| use_existing = true | ||
| owners = local.owners | ||
| } | ||
|
|
||
| resource "azuread_application_password" "portal" { | ||
| application_id = azuread_application.portal.id | ||
| display_name = "portal-${var.deployment_env}" | ||
| rotate_when_changed = { | ||
| rotation = time_rotating.rotation.id | ||
| } | ||
| } | ||
|
|
||
| resource "time_rotating" "rotation" { | ||
| rotation_days = 180 | ||
| } | ||
|
|
||
| resource "random_password" "auth" { | ||
| length = 48 | ||
| special = false | ||
| } | ||
|
|
||
| resource "kubernetes_secret_v1" "portal" { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The README calls this out, but I think we should just fix it instead of documenting it. kubernetes_secret_v1 owns the whole portal-secrets object, so the SMTP/Discord/Windows keys added by hand get wiped on every apply and every rotation.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see nothing wrong with any changes made by hand getting overridden. In the spirit of Infrastructure as code, the state of the infrastructure should be defined... well... in code, so anything done to cause the infrastructure to drift should be reconciled by the IaC deployments. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fair, but then the README shouldnt tell people to add the SMTP/Discord/Windows keys to that secret by hand. either pull them into terraform too (sensitive vars or a key vault data source) or give them their own secret the chart also mounts. right now the doc is describing drift on purpose
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤦♂️Oh, good point. I get what you mean now. I think later on, the other secrets will be pulled from Vault and populated in the secret. (That won't be part of this PR though. 😉) |
||
| metadata { | ||
| name = "portal-secrets" | ||
| namespace = var.kubernetes_namespace | ||
| } | ||
|
|
||
| type = "Opaque" | ||
|
|
||
| data = { | ||
| BETTER_AUTH_SECRET = random_password.auth.result | ||
| MICROSOFT_CLIENT_ID = azuread_application.portal.client_id | ||
| MICROSOFT_CLIENT_SECRET = azuread_application_password.portal.value | ||
| MICROSOFT_TENANT_ID = data.azuread_client_config.current.tenant_id | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| terraform { | ||
| backend "azurerm" { | ||
| resource_group_name = "acm-hybridcloud" | ||
| storage_account_name = "acmhybridstore" | ||
| container_name = "acm-terraform-state" | ||
| key = "acm-member-portal.tfstate" | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| deployment_env = "prod" | ||
|
|
||
| redirect_uris = [ | ||
| "https://portal.acmuic.org/api/auth/callback/microsoft", | ||
| ] | ||
|
|
||
| logout_uris = [ | ||
| "https://portal.acmuic.org", | ||
| ] | ||
|
|
||
| kubernetes_namespace = "acm-portal" | ||
|
|
||
| additional_owner_ids = [ | ||
| "21ec2c63-534d-4cfb-bc43-83773ce40e54", # terraform-svc | ||
| "920bf583-cb9b-4365-a61e-bb33988c5484" # clee231 | ||
| ] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| output "application_id" { | ||
| value = azuread_application.portal.id | ||
| } | ||
|
|
||
| output "client_id" { | ||
| value = azuread_application.portal.client_id | ||
| } | ||
|
|
||
| output "service_principal_object_id" { | ||
| value = azuread_service_principal.portal.object_id | ||
| } | ||
|
|
||
| output "secret_name" { | ||
| value = kubernetes_secret_v1.portal.metadata[0].name | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just checking, what happens to the running pods when the 180 day rotation fires? The password gets replaced (destroy then create) and the Secret updates, but the pods keep the old env value until they restart. The old password is gone at that point so Microsoft login would break until someone does a rollout.
▎
▎ Either create_before_destroy = true on the password so both are valid during the swap, or a Reloader / checksum annotation on the deployment so it picks up Secret changes. Probably worth a line in the README either way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you an AI bot? 😮
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
im hurt Chase
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It just doesn't seem like something you'd write, so I was confused. 🤔
But anyways, after the 180 days, the password will get rotated after the 180 day rotation occurs. The expectation is the operator will handle the credential switchover pretty shortly after the run, so we don't really need to have both passwords valid at a time.
Eventually, we'll get this automated, so we will soon™️be able to have a closed loop with no human intervention.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
:( thats how i write though