I recently needed to copy a few small text files into a container running in AWS Fargate. I assumed it would be an easy matter of copying it through the terminal session, but that is, unfortunately, not the case.
I was accessing the containers via the aws ecs execute-command --interactive --comand "bash" mechanism. If you’re not familiar, this is a quick way to get a terminal inside an ECS container running in any cluster. It’s especially helpful when you have containers that do not receive incoming traffic from the internet. It is almost like SSH, but it is not exactly SSH. And one of the limitations is that there is no equivalent to scp. There is also no ability to forward ports, so you can’t use it to access SSH either.
You could assign a public IP to every container you run so that it’s accessible via SSH. But, why would you? Not only does it cost more (though only a little bit), it is increasing the attack surface of your system.
The easiest way to get a couple of text files onto an ECS container is to copy and paste their contents into a file through the terminal window. This only works with plain text files, but for scripts this is usually sufficient. For example:
cat >myscript.sql <<EOF
paste your contents here
EOF
I tried this at first, but about 5 files in I started to wonder if I was still pasting the right contents into the right file. There is a kind of mindless haze the sets in when doing repetitive operations, and it invites mistakes. I’ve also got a nurtured sense of laziness that pushes me to find new ways to solve problems.
After a bit of fiddling, I discovered that it’s possible to compress a bunch of files into a giant base64 blob, which can easily be pasted into the terminal session, and unpacked inside it. This means you’re only doing one paste operation, and binary files can be transmitted too! Here is how to do it:
First, you should connect to the destination machine. You can use a command like this with the AWS CLI. Note that you need to be authenticated and configured to use the correct AWS region first. You’ll also need to look up the task id, which can also be done from the command line, but I find the AWS console faster.
aws ecs execute-command --cluster mycluster --interactive --command "bash" --container mydatabase --task 6932e628a2ca4a15b53d9e25d9bd3193
The main reason to do this first is so that you only have to copy and paste the base64 blob once. Guess how many times I’ve copied the blob, only to realize I need to use the clipboard for the task id, and then have to select and copy the blob again afterwards?
Next, from a different terminal window on the source machine, you use tar and base64 to convert the desired files into a blob of base64:
cd location/of/files
tar -czf - *.sql | base64
This will dump all the content to the terminal. Depending on your environment, you may be able to send it directly to a clipboard. I can’t figure out how to do it in my WSL environment, so I just select it and copy it manually. It may help to use a clear command first so that you can find the start of the blob more easily.
Back on the destination machine, stick the base64 data into a temporary file:
cat >pasted.b64 <<EOF
paste contents here
EOF
Now you can unpack the contents into the current directory with this command, and continue with your task.
base64 -d pasted.b64 | tar -xzf -
You could also use the same technique in reverse to move a bunch of text files from an ECS container back to your local machine. Or even between two different ECS containers.
Or maybe some nice person will write an aws-ecs-scp command to do all the hard work for us.