Powershell script to access Amazon S3 as a Cognito userpool user

Continuing from my last blog, I started accessing the S3 bucket as an Amazon Cognito user using PowerShell. After all, the idea was to create a script to access S3 resources using access/refresh tokens.

First, I thought it is going to be trivial. Like many other APIs AWS PowerShell module implementations will have one-to-one commandlets to pretty much convert the CLI script to PowerShell – well, I was wrong.

 
To start with, commands are spread across various modules; each has very similar sounding API commands that does different things. Then, it was challenging to find out where to send the REST request – still, if you search the web, it is apparent that there is confusion on should it be sent to the common amazon gateway or your Coginto endpoint. It was the former - I had to ran a good old network trace to find out where the CLI command sends the request

 
Another challenge was figuring out how to send data. I hadn’t used AWS gateway with HTTP posts before, so it took a while to figure out the exact methods. Parameters are spread across the header and post body.

 
After that it is reasonably straightforward. Once you get an id_token for the Cognito user,
Use API 

com.amazonaws.cognito.identity.model.AWSCognitoIdentityService.GetId 

to get the id pool identity. This is not strictly necessary as it is shown under identity browser in the identity pool, but this call makes it more generalized.

Then use the following API to get the credentials.

com.amazonaws.cognito.identity.model.AWSCognitoIdentityService.GetCredentialsForIdentity 

Send these requests to the common Cognito gateway for your region, for example,

https://cognito-identity.<region>.amazonaws.com

Once the credentials are in hand, use read-s3object Commandlet to read S3 Objects..

Finally, this was the script.

$id_tkn = "<id_token>"
$identityPoolId = "<region>:<GUID>"
$id_provider = "cognito-idp.<region>.amazonaws.com/<userpool_id>"

$idbody = @"
{
"IdentityPoolId": "$identityPoolId",
 "Logins": {
     "$id_provider":"$id_tkn"
     }
}
"@

$idheaders=@{'X-AMZ-TARGET'='com.amazonaws.cognito.identity.model.
  AWSCognitoIdentityService.GetId';  'CONTENT-TYPE' = 'application/x-amz-json-1.1' } 
$credheaders=@{'X-AMZ-TARGET'='com.amazonaws.cognito.identity.model.
  AWSCognitoIdentityService.GetCredentialsForIdentity';  
  'CONTENT-TYPE' = 'application/x-amz-json-1.1'} 


$id = Invoke-RestMethod -uri https://cognito-identity.<region>.amazonaws.com 
  -Method Post -Body $idbody -Headers $idheaders

$IdentityId = $id.IdentityId

$credBody= @"
{
  "IdentityId": "$IdentityId",
  "Logins": {
         "$id_provider":"$id_tkn"
     }
}
"@

$cred = Invoke-RestMethod -uri https://cognito-identity.<region>.amazonaws.com 
  -Method Post -Body $credbody -Headers $credheaders

$accKey = $cred.Credentials.AccessKeyId
$secretKey = $cred.Credentials.SecretKey
$sessionTkn = $cred.Credentials.SessionToken

read-s3object -bucketname <bucket or accesspoint name> -key <item_key> 
  -File  <local file name to store> -AccessKey $accKey 
  -SecretKey $secretKey -SessionToken $sessionTkn

Getting AWS Cognito user pool users to access different S3 resources

I started working with AWS and oAuth. Compared to Azure, finding information on AWS is quite challenging. There are many docs in AWS, but they are auto-generated like isolated explanations than answering the questions you have when doing a real-life solution.

My task was to allow S3 access to users who don't have AWS accounts, using oAuth. Amazon oAuth infrastructure is in the Amazon Cognito service.

Authorization

Once the identity and User pools are created, you need to add an app client for the user pool. App client controls the attributes of keys such as the lifetime. Refresh, access, and Id tokens have unique values for the validation period. Then calling the authorize and token endpoints with appropriate parameters provide refresh token and access token. The scope should include the parameter OpenId to receive an OpenID token from the refresh/authorize keys. The access token is no good to access S3 as it doesn't contain the token issuer's details. Different identity providers such as Google and Facebook are supported by the Cognito identity pool, but I chose to use the Cognito identity provider. The identity provider does not make much of a difference to the process. Cognito identity doesn't have any deep relationship with S3 but treated as any other provider.

Identity

Once the user/identity pool is created, auth and unauth roles from the identity pool are visible in the IAM console under roles (Primary reason I made them in the same AWS account as S3). You can create granular policies in the policy section, defining who can access what resources, at what level by each role. Though the policies can be tied directly to the buckets or parts of it, I created an access point that makes managing more comfortable.

Accessing S3 resources using aws cli

I had to work for days on this. Though the AccessToken is granted from Cognito, there was no interface to add it to S3 APIs. S3 APIs look for access keys and a secret, usually associated with the IAM users. Finally, it turned out asking for keys by providing an OpenID does the trick. If the string "OpenID" is in the scope when asking for authorization, Cognito sends an OpenID with the following format. 

{

  "kid": "lXY ...... =",

  "alg": "RS256"

}

JSON {

  "at_hash": "4y9WXD5LhlIBA1jSNMnjYA",

  "sub": "c4 ,...... 88",

  "email_verified": true,

  "iss": "https://cognito-idp.<region>.amazonaws.com/<UserPoolId",

  "phone_number_verified": true,

  "cognito:username": "c3.....3",

  "given_name": "name",

  "aud": "....",

  "token_use": "id",

  "auth_time": 1613947027,

  "nickname": "<name>",

  "phone_number": "<PhoneNumber>",

  "exp": 1614557229,

  "iat": 1614553629,

  "family_name": "<Surname>",

  "email": "<email>"

}

To get the temporary ids, Cognito needs the user pool id. Though this is available in Console when browsing identities, asking for it by sending the key makes it more generalized. When sending this to retrieve a temporary credential, the key is sent in the logins option, which has the format "provider=key". Use awscli command getid to get temp credentials. aws cognito-identity get-id --identity-pool-id <identity pool id>> --logins "cognito-idp.<region>.amazonaws.com/<identity pool id>=<id token>" Which sends the user pool id in the format,

{

    "IdentityId": "<user pool id>"

}

Now we have everything required to ask for the temporary key using get-credentials-for-identity.

aws cognito-identity get-credentials-for-identity --identity-id "<user pool id>" --logins "cognito-idp.<region>.amazonaws.com/<identity pool id>=<id token>"

Response is in the format,

{

    "IdentityId": "<user pool id>",

    "Credentials": {

        "AccessKeyId": "ASIAOKHKDLENZPN72UMU",

        "SecretKey": "TKxoZruHTjhyD87kjDjhjHSJ7uj78zB9rXpbMZ2/",

        "SessionToken": "IQoJb3J ……… H+FMZ4azHYaoo26uXB3A1dBLWe4fqLNG03RCVTMahodcnPs+2LiBzA/vlPf/Zo8=",

        "Expiration": "2021-03-01T14:22:23+13:00"

    }

}

There are many ways to provide the credentials, but I chose the most straightforward method - setting them as environment variables. When aws APIs look for credentials, environment variables has the highest priority.

Set AWS_ACCESS_KEY_ID=<accessKeyId>

Set AWS_SECRET_ACCESS_KEY=<SecretKey>

Set AWS_SESSION_TOKEN=<Session Token>

Now, use the list command to list the contents of the s3 access point.

> aws s3 ls s3://<access point name>

                           PRE testdata/

2020-11-20 13:00:42       1011 test.csv

2020-11-20 13:00:42       2338 test - (Copy).csv

Download a file using cp.

aws s3 cp s3://<access point arn>/test.csv c:\temp

download: s3://<arn>/test.csv to ..\..\temp\test.csv