TLDR

If you just want to get started asap with a service principal and azure with a client
secret and permissive RBAC settings, then just run the following commands (redacted as required)

app_display_name="<your_app_name>"
subscription_id="<your_subscription_id>"
az login
az account set -s "${subscription_id}"
az ad app create --display-name "${app_display_name}"
app_obj_id=$( \
  az ad app list \
    --display-name "${app_display_name}" \
    --query [].objectId \
    --output tsv
)
az ad sp create --id "${app_obj_id}"
spn_app_id=$( \
  az ad sp list \
    --display-name "${app_display_name}" \
    --query [].appId \
    --output tsv
)
az ad sp credential reset --name "${spn_app_id}"
spn_obj_id=$( \
  az ad sp list \
    --display-name "${app_display_name}" \
    --query [].objectId \
    --output tsv
)
tenant_id=$( \
  az ad sp list \
    --display-name "${app_display_name}" \
    --query [].appOwnerTenantId \
    --output tsv
)
az role assignment create \
  --assignee-object-id "${spn_obj_id}" \
  --assignee-principal-type "ServicePrincipal" \
  --role "Contributor" \
  --scope "/subscriptions/${subscription_id}"

# Test the newly created service principal as follows:
az logout
az login \
  --service-principal \
   --username "${spn_app_id}" \
   --password "<your_service_principal_password>" \
   --tenant "${tenant_id}"

Why use a Service Principal anyways

Permalink to “Why use a Service Principal anyways”#

When it comes to running some commands adhoc off the command line, a human user account
based on a user principal account is fine. But for any serious use case,
this would not qualify as a sustainable devops practice. Typically this is done by calling
the az login command from a terminal which invokes a web-based login process in the background.
For automation, a service principal account is required to run tasks such as:

• Continuous Integration tasks (eg blob storage access)
• Continuous Deployment tasks (eg deployment of a vm)
• Infrastructure as Code Deployments (eg using terraform)
• Running scheduled tasks
• Accessing Azure Key vault or Azure Blob storage from a running application in
  application code using the Azure SDKs

Anatomy of a Service Principal

Permalink to “Anatomy of a Service Principal”#

A service principal does not exist in isolation. In fact, a service principal is a
'security principal' or identity that represents an active directory application in a given
tenant. By default, a security principal will be created in the default tenant along with
the application object. As azure active directory is multi-tenant, further service principals
can be created for additional tenants should the requirement arise.

Service Principal and authentication

Permalink to “Service Principal and authentication”#

Authentication with a service principal can be through using

• A client secret
• A client certificate (X.509 self-signed certificate)

Generally, I prefer the creation of an azure service principal with a client certificate which
has been self-signed.

From a security perspective, it is vital to keep both secrets and certificates secure. This
can be done by encrypting the data with mozilla sops.

Azure Terraform Provider

Permalink to “Azure Terraform Provider”#

Terraform is a highly declarative language for defining infrastructure and
I normally prefer it for creating all kinds of infrastructure.

• Terraform still requires a service principal to get started - chicken and egg situation.

• The service principal for the azuread_service_principal
  terraform module requires the User Account Administrator role. From a security perspective, using
  such a highly priveleged service principal requires careful scrutiny.

Creating an Azure Application

Permalink to “Creating an Azure Application”#

Prerequisites

Permalink to “Prerequisites”#

• Azure CLI - The installation instructions of az cli on ubuntu can be found here

Login with a user principal

Permalink to “Login with a user principal”#

az login

This will return the following information revealing the user, default tenant and
default subscription information.

[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "<tenant_id>",
    "id": "<subscription_id>",
    "isDefault": true,
    "managedByTenants": [],
    "name": "<subscription_name>",
    "state": "Enabled",
    "tenantId": "<tenant_id>",
    "user": {
      "name": "<user_principal>",
      "type": "user"
    }
  }
]

Ensure the Correct Azure Subscription is selected

Permalink to “Ensure the Correct Azure Subscription is selected”#

If you need to use a specific subscription other than the default one when logged in, it can
be set as follows

subscription_id="<your_subscription_id>"
az account set -s "${subscription_id}"

Note: You can check what subscriptions are available by running az account list.

Create the azure application

Permalink to “Create the azure application”#

Docs for az ad app create

app_display_name="spn-for-ci-2"
az ad app create \
  --display-name "${app_display_name}"

The extract below shows some of the important fields from the http response including the
appId and objectId.

{
  "appId": "<app_id>",
  "displayName": "spn-for-ci-1",
  "oauth2AllowIdTokenImplicitFlow": true,
  "oauth2RequirePostResponse": false,
  "objectId": "<app_object_id>",
  "objectType": "Application"
}

Note: You can check what apps are already created by running

az ad app list
az ad app list --display-name "${app_display_name}"

Creating the Service Principal

Permalink to “Creating the Service Principal”#

The service principal is created next and associated with the previously created app. The
appId or objectId can be used to assign the service principal

Docs for az ad spn create

app_display_name="spn-for-ci-2"
app_obj_id=$( \
  az ad app list \
    --display-name "${app_display_name}" \
    --query [].objectId \
    --output tsv
)
az ad sp create --id "${app_obj_id}"

Extract from the json response:

{
  "accountEnabled": "True",
  "appDisplayName": "spn-for-ci-1",
  "appId": "<app_id>",
  "appOwnerTenantId": "<tenant_id>",
  "appRoleAssignmentRequired": false,
  "appRoles": [],
  "displayName": "spn-for-ci-1",
  "objectId": "<object_id>",
  "objectType": "ServicePrincipal",
  "servicePrincipalNames": ["<spn_name_1>"],
  "servicePrincipalType": "Application"
}

Credentials creation with a client secret

Permalink to “Credentials creation with a client secret”#

So far no credentials have been created for the service principal.
Let's take a look...

Docs for az ad sp credential list

app_display_name="spn-for-ci-2"
spn_app_id=$( \
  az ad sp list \
    --display-name "${app_display_name}" \
    --query [].appId \
    --output tsv
)
az ad sp credential list --id "${spn_app_id}"

The response is an empty list which confirms roles are not yet assigned

[]

Create new credentials with a client secret

az ad sp credential reset --name "${spn_app_id}"

The response is as follows:

{
  "appId": "<app_id>",
  "name": "<app_name>",
  "password": "<app_password>",
  "tenant": "<tenant_id>"
}

Once these credentials are returned, you will need to store them somewhere secure for
later retrieval, ideally, in an encrypted format.

Running the following command again will reveal the credentials:

az ad sp credential list --id "${spn_app_id}"

Note: If you are a user of terraform azure provider,
the credentials above correspond to the following naming in terraform:

Azure	Terraform	Terraform environment variable
appId	client_id	ARM_CLIENT_ID
password	client_secret	ARM_CLIENT_SECRET
tenant	tenant_id	ARM_TENANT_ID

Credentials creation with a client certificate

Permalink to “Credentials creation with a client certificate”#

Creating a certificate using the azure cli

Permalink to “Creating a certificate using the azure cli”#

The easy way to do this is to have the azure cli create a certificate
for you. Let's append new credentials by creating a public cert and private key
with the azure cli.

app_display_name="spn-for-ci-2"
spn_app_id=$( \
  az ad sp list \
    --display-name "${app_display_name}" \
    --query [].appId \
    --output tsv
)
az ad sp credential reset \
  --name "${spn_app_id}" \
  --append \
  --create-cert \
  --years 3

Note: With the --append argument, the previous secret based credentials will not be overridden.
Omit --append if you want to use a client certificate only and override previous credentials.

The output is as follows:

{
  "appId": "<app_id>",
  "fileWithCertAndPrivateKey": "<path_to_pem",
  "name": "",
  "password": null,
  "tenant": ""
}

The value for fileWithCertAndPrivateKey contains the path to the public certificate and the private key.
This file should be stored somewhere safely and ideally in an encrypted format. It is the key
to the kingdom. Although, we will be reducing the dominion of that kingdom later by
employing RBAC and setting roles on the service principal.

Creating a certificate with your own certificate authority

Permalink to “Creating a certificate with your own certificate authority”#

You can also create your own certificate and private key using your own certificate authority
for creating the service principal credentials.

In this guide, I will create a certificate authority to create the azure service principal
certificate. If you already have your own certificate authority then you can skip this
step. Basically, this means using your own PKI (public key infrastructure).

# Approximately 10 years before expiry (can be renewed then)
cert_authority_data=$(cat <<EOF
{
  "CA": {
      "expiry": "87600h",
      "pathlen": 0
  },
  "CN": "Company Name Certificate Authority",
  "key": {
    "algo": "rsa",
    "size": 4096
  },
  "names": [
    {
      "C": "IRL",
      "L": "Limerick",
      "ST": "Co. Limerick",
      "O": "Some Company Name Ltd.",
      "OU": "Devops"
    }
  ]
}
EOF
)
echo "${cert_authority_data}" | cfssl gencert -initca - | cfssljson -bare ca -

You can keep the files ca.pem and ca-key.pem for further uses of as your certificate authority.
The ca-key.pem is the private key ancd should be kept very safe, ideally in an encrypted format.

The next step is to create the new service principal private key and certificate

cert_data=$(cat <<EOF
{
  "CN": "spn-for-ci-1",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "IRL",
      "L": "Limerick",
      "ST": "Co. Limerick",
      "O": "Some Company Name Ltd.",
      "OU": "Devops"
    }
  ]
}
EOF
)

cat <<EOF > ca-config.json
{
  "signing": {
    "default": {
      "expiry": "26280h",
      "usages": [
        "signing",
        "key encipherment",
        "server auth",
        "client auth"
      ]
    }
  }
}
EOF

echo "${cert_data}" |
cfssl gencert \
  -ca=ca.pem \
  -ca-key=ca-key.pem \
  -config=ca-config.json - | cfssljson -bare

As a result of running the last command, a private key file cert-key.pem and public cert cert.pem
will be generated. Be sure to store these files away, in a safe location, ideally encrypted. Azure
requires having these two files concatenated together to login as a service principal. The login
process is demonstrated later

Now we are ready to create the service principal with the self-signed certificate as follows:

app_display_name="spn-for-ci-2"
spn_app_id=$( \
  az ad sp list \
    --display-name "${app_display_name}" \
    --query [].appId \
    --output tsv
)
# cat cert-key.pem > az-cert.pem && cat cert.pem >> az-cert.pem
az ad sp credential reset \
  --name "${spn_app_id}" \
  --append \
  --cert "@cert.pem"

Given the complexity of creating your own public key infrastructure and generating the certs, it
is probably easier to just create a certificate using the azure cli as outlined earlier.

Note: If you are a user of terraform azure provider,
the credentials above correspond to the following naming in terraform:

Azure	Terraform	Terraform environment variable
appId	client_id	ARM_CLIENT_ID
fileWithCertAndPrivateKey	client_certificate_path	ARM_CLIENT_CERTIFICATE_PATH
tenant	tenant_id	ARM_TENANT_ID

Roles assignment and RBAC

Permalink to “Roles assignment and RBAC”#

Role Based Access Control allows devops and software engineers to create service
principals following the least privilege principle. Here are two ways to approach
assigning roles to an azure service principal:

• Assign built-in roles
• Assign a custom role definition

After the service principal has been associated to the app in the default
tenant, no roles have yet been assigned to the service principal.
Roles are the way in which access control works in Azure. Let's take a look at
the roles assigned so far.

Docs for az role assignment list

app_display_name="spn-for-ci-2"
spn_obj_id=$( \
  az ad sp list \
    --display-name "${app_display_name}" \
    --query [].objectId \
    --output tsv
)
az role assignment list --assignee "${spn_obj_id}"

The response is an empty list which confirms roles are not yet assigned

[]

Assigning a built-in role to a service principal

Permalink to “Assigning a built-in role to a service principal”#

subscription_id="<your_subscription_id>"
app_display_name="spn-for-ci-2"
spn_obj_id=$( \
  az ad sp list \
    --display-name "${app_display_name}" \
    --query [].objectId \
    --output tsv
)
az role assignment create \
  --assignee-object-id "${spn_obj_id}" \
  --assignee-principal-type "ServicePrincipal" \
  --role "Contributor" \
  --scope "/subscriptions/${subscription_id}"

Notes:

• The Contributor
  built-in role is highly permissive - avoid using it when possible. It would be better to assign the minimal roles necessary.
  For example, if an application only requires read access to an azure blob storage container, then the Storage Blob Data Reader
  is all that is required. Assigning multiple restrictive built-in roles is also possible and better
  than applying the highly permissive Contributor
  role.

• The --scope argument allows one to reduce the access levels of the service principal even further. It can be restricted
  to specific subscription(s) and even more fine grained to specific resource group(s).

By using a sensible combination of roles and scopes, the overall security of operations in the cloud is enhanced. In the unlikely event
that a service principal becomes comprimised, the potential for malicious activitty is reduced to the smallest range of resources.

Assigning a custom role definition to a service principal

Permalink to “Assigning a custom role definition to a service principal”#

An entirely custom role can be built up from scratch and assigned to a service principal.
This is a very powerful approach to RBAC for a service principal because it gives the cloud
operator fine-grained control of what permissions are granted.

custom_role=$(cat <<EOF
{
  "Name": "My Storage Read Role",
  "IsCustom": true,
  "Description": "Can read from storage containers.",
  "Actions": [
    "Microsoft.Storage/*/read",
    "Microsoft.Authorization/*/read",
    "Microsoft.Resources/subscriptions/resourceGroups/read",
  ],
  "NotActions": [],
  "DataActions": [],
  "NotDataActions": [],
  "AssignableScopes": [
    "/subscriptions/${subscription_id}",
  ]
}
EOF
)

az role assignment create \
  --assignee-object-id "${spn_obj_id}" \
  --assignee-principal-type "ServicePrincipal" \
  --role "Contributor" \
  --scope "/subscriptions/${subscription_id}"

Logging in

Permalink to “Logging in”#

Once the credentials creation step and rbac assignment steps are complete, the service principal is
then ready to use and it can be tested to verify that it is working correctly.

While still logged in as a user principal, you can get set some of the variables in
the terminal before running an azure logout.

app_display_name="spn-for-ci-2"
spn_app_id=$( \
  az ad sp list \
    --display-name "${app_display_name}" \
    --query [].appId \
    --output tsv
)
tenant_id=$( \
  az ad sp list \
    --display-name "${app_display_name}" \
    --query [].appOwnerTenantId \
    --output tsv
)

At this stage, logout with your user principal account.

az logout

Logging in with a client secret

Permalink to “Logging in with a client secret”#

az login \
  --service-principal \
   --username "${spn_app_id}" \
   --password "<your_service_principal_password>" \
   --tenant "${tenant_id}"

Logging in with an azure generated client cert

Permalink to “Logging in with an azure generated client cert”#

When you ran the az cli command to create credentials to a client cert as
outlined in above, the json
response included a key pair "fileWithCertAndPrivateKey": "<path_to_pem>".
This file path contains the private key and public certificate in a single file.

To login, just pass the file path in the password field of the az login command as
follows:

az login \
  --service-principal \
   --username "${spn_app_id}" \
   --password /path/to/azure-generated-key-and-cert.pem \
   --tenant "${tenant_id}"

Logging in with a self-signed client cert

Permalink to “Logging in with a self-signed client cert”#

Assuming that cert-key.pem is the private key and cert.pem is the public
certificate, azure requires that those certs are concatenated into
a single file and the path of the newly created concatenated cert file
passed in the password field for login.

az login \
  --service-principal \
   --username "${spn_app_id}" \
   --password "$(cat cert-key.pem > /tmp/crt.pem && cat cert.pem >> /tmp/crt.pem && echo /tmp/crt.pem)" \
   --tenant "${tenant_id}"
rm /tmp/crt.pem

Now that you have your service principal setup, it's time to use it for some terraform
automation.