Part 5

Sorry for the delay between part 4 and 5! I was at the Microsoft MVP Summit this past week and didn’t have time to devote towards updating. This series is nearing completion with just a few more parts to go.

In Part 4 we published the AWS Lambda and created the AWS KMS Key that will be used for encrypting and decrypting secrets. In Part 5 we will configured the AWS CodeCommit repository trigger to invoke the AWS Lambda and encrypt our secrets to store in in the cc2af.yml file.

Series Table of Contents

Grant AWS CodeCommit Repository Execution Permission the AWS Lambda

With the AWS Lambda created we can now link the AWS CodeCommit repository up to execute it. However, before we do that we need to grant the CodeCommit repository rights to invoke the Lambda.

# Grant the CodeCommit Repository access to execute the Lambda function
$Params = @{
    Action       = 'lambda:InvokeFunction'
    FunctionName = $Settings.LambdaName 
    Principal    = 'codecommit.amazonaws.com'
    SourceArn    = $AWSAssets.CCRepository.Arn
    StatementId  = $Settings.CCLambdaPolicyStatementId
}
$AWSAssets['CCLambdaPermission'] = Add-LMPermission @Params -Force

Then we need to verify it with Pester:

# Validate the policy has been applied
Describe "CodeCommit Lambda policy" {
    It "Was successfully added" {
        $policy = Get-LMPolicy -FunctionName $Settings.LambdaName | 
            Select-Object -ExpandProperty Policy |
            ConvertFrom-Json
        $statement = $policy.Statement[0]

        $policy.Statement | Should -HaveCount 1
        $statement.Sid | Should -BeExactly $Settings.CCLambdaPolicyStatementId
        $statement.Effect | Should -BeExactly 'Allow'
        $statement.Principal.Service | Should -BeExactly 'codecommit.amazonaws.com'
        $statement.Condition.ArnLike.'AWS:SourceArn' | Should -BeExactly $AWSAssets.CCRepository.Arn
    }
}

And it it was successful you should see:

Add AWS CodeCommit Repository Trigger to Execute AWS Lambda

We want our commits to the AWS CodeCommit repository to trigger an automatic deployment to Azure Functions. To do that we need need to add the trigger to the AWS CodeCommit repo to invoke our AWS Lambda. The following code creates a trigger when any action is taken on the master branch of the repository:

# Add a Repository Trigger to the CodeCommit Repository to Execute the AWS Lambda function
$repositoryTrigger = [Amazon.CodeCommit.Model.RepositoryTrigger]::New()
$repositoryTrigger.Branches.Add('master')
$repositoryTrigger.DestinationArn = $AWSAssets.PublishedLambda.FunctionArn
$repositoryTrigger.Events = 'all'
$repositoryTrigger.Name = $Settings.CCTriggerName
$Params = @{
    RepositoryName = $Settings.CCRepositoryName
    Trigger        = $repositoryTrigger 
    Force          = $true
}
$AWSAssets['CCRepositoryTrigger'] =  Set-CCRepositoryTrigger @Params

Next we verify the repository trigger was added with Pester:

# Validate the repository trigger
describe "CodeCommit Repository Trigger" {
    it "Was successfully added." {
        $triggers = Get-CCRepositoryTrigger -RepositoryName $Settings.CCRepositoryName

        $triggers.Triggers[0].Branches | Should -HaveCount 1
        $triggers.Triggers[0].Branches[0] | Should -BeExactly 'master'
        $triggers.Triggers[0].DestinationArn | Should -BeExactly $AWSAssets.PublishedLambda.FunctionArn
        $triggers.Triggers[0].Name | Should -BeExactly $Settings.CCTriggerName
    }
}

Which should result in:

At this point, any commit to the master branch of the PBnC CodeCommit repository will trigger an invocation of the TriggerAzureFunctionDeployment Lambda which will then trigger the git deployment on the Azure Functions Web App. If we made a commit now, without a cc2af.yml, the deployment will fail because it does not have the required configuration.

Functions for Encrypting and Decrypting Strings with AWS KMS Key

Before we can create the cc2af.yml file, we need a few helper functions to encrypt and decrypt secrets. We will need to keep these functions around for future use should any of our secrets (the AWS IAM User HTTPS Git Credentials password and the Azure Functions Web Service Deployment credentials password) change. We also need them to create the encrypted strings to store in the cc2af.yml.

KMS does not encrypt and decrypt strings directly. Instead, KMS works with binary data streams. This makes encrypting and decrypting strings a bit inconvenient in PowerShell as the AWSPowerShell module does not provide an easy to use function to do this. Also, when the decrypted binary is returned, it is not a binary representation of a string. The string we want to encrypt needs to be converted to binary, the binary then encrypted with KMS, the returned binary then needs to be Base64 encoded. This means that what we get is a Base64 representation of KMS encrypted binary.

This is the function for encryption:

# A function to encrypt a string with KMS and then return a base64 encoded string representation
function ConvertTo-Base64KMSEncryptedString {
    [CmdletBinding()]
    param (
        [Parameter(
            Mandatory = $true,
            ValueFromPipeline = $true
        )]
        [String[]]
        $String,

        [Parameter(
            Mandatory = $true
        )]
        [string]
        $KeyId,

        [hashtable]$EncryptionContext
    )
    
    process {
        foreach ($SourceString in $String) {
            $byteArray = [System.Text.Encoding]::UTF8.GetBytes($SourceString)
            $stringStream = [System.IO.MemoryStream]::new($ByteArray)
            try {
                $Params = @{
                    KeyId = $KeyId 
                    Plaintext = $stringStream 
                    ErrorAction = 'Stop'
                }
                if ($EncryptionContext) {
                    $Params['EncryptionContext'] = $EncryptionContext
                }
                $KMSResult = Invoke-KMSEncrypt @Params

                [System.Convert]::ToBase64String($KMSResult.CiphertextBlob.ToArray())
            }
            finally {
                if ($stringStream) { $stringStream.Dispose() }
                if ($KMSResult.CiphertextBlob) { $KMSResult.CiphertextBlob.Dispose() }
            }
        }
    }
}

Decryption just runs the opposite direction. We need to take a Base64 string, convert it to binary, decrypt the binary, then convert the returned decrypted binary to a string.

# A function to decrypt a base64 representation of a string encrypted by KMS.
function ConvertFrom-Base64KMSEncryptedString {
    [CmdletBinding()]
    param (
        [Parameter(
            Mandatory = $true,
            ValueFromPipeline = $true
        )]
        [String[]]
        $EncryptedString,

        [hashtable]$EncryptionContext
    )
    
    process {
        foreach ($SourceString in $EncryptedString) {
            try{
                $byteArray = [System.Convert]::FromBase64String($SourceString)
            }
            Catch {
                Write-Error -ErrorRecord $_
                continue
            }
            $stringStream = [System.IO.MemoryStream]::new($byteArray)
            try {
                $Params = @{
                    CiphertextBlob = $stringStream 
                    ErrorAction = 'Stop'
                }
                if ($EncryptionContext) {
                    $Params['EncryptionContext'] = $EncryptionContext
                }
                $KMSResult = Invoke-KMSDecrypt @Params

                $reader = [System.IO.StreamReader]::new($KMSResult.Plaintext)
                $reader.ReadToEnd()
            }
            finally {
                if ($reader){ $reader.Dispose() }
                if ($stringStream){ $stringStream.Dispose() }
            }
        }
    }
}

One unfortunate problem is that the Base64 string representations of the encrypted binary that are returned are rather lengthy. I tried several compression techniques, including gzip, but they did not yield results that made them worth the complexity of implementation. One idea would be combine gzip and base85 encoding. However, AES encrypted binary data doesn’t necessarily compress well, and base85 only offers a 7% reduction in string size over base64. If anyone has a good idea to make these encrypted strings more compact while still using KMS, please let me know.

It’s important to note that the AWS Lambda has a C# implementation of decryption:

public static string ConvertFromEncryptedBase64 (string encryptedBase64)
{
    string result;
    using (var kms = new AmazonKeyManagementServiceClient())
    {
        var response = kms.DecryptAsync(new DecryptRequest(){
            CiphertextBlob = new MemoryStream(Convert.FromBase64String(encryptedBase64))
        }).GetAwaiter().GetResult();
        using (TextReader reader = new StreamReader(response.Plaintext))
        {
            result = reader.ReadToEnd();
        }
    }
    return result;
}

Also, you do not require the Key ID to decrypt data with KMS. I’m guessing the encrypted binary includes information on the key used to encrypt it. You (or the role your code assumes) just need to have decryption permissions on the required key.

Encrypt Secrets with AWS KMS

With our helpful functions available we can now encrypt our secrets.

# Encrypt the CodeCommit Git User Password and base64 encode it
$AWSAssets['EncryptedGitPassword'] = $AWSAssets.CCGitUserCredentials.GetNetworkCredential().password |
    ConvertTo-Base64KMSEncryptedString -KeyId $AWSAssets.KMSKey.KeyId

# Encrypt the CodeCommit Git User Password and base64 encode it
$AzureAssets['EncryptedWebAppPassword'] = $AzureAssets.WebAppUserPwd |
    ConvertTo-Base64KMSEncryptedString -KeyId $AWSAssets.KMSKey.KeyId

Part 5 End

That’s it for Part 5. Part 6 we will create the cc2af.yml file and perform out first Azure Function deployment from AWS CodeCommit.

Get-PowerShellBlog

2018-03-10

Peanut Butter and Chocolate: Azure Functions CI/CD Pipeline with AWS CodeCommit (Part 5 of 6)