16 Jun 2022

docker exec: Passing in environment variables

I’ve been working on an Apache Pinot recipe showing how to ingest data from S3 and I needed to pass in my AWS credentials to the docker exec command that I was running. It wasn’t difficult to do, but took me a little while to figure out.

Figure 1. docker exec: Passing in environment variables

The command that I was running looked like this:

docker exec \
  -it pinot-controller bin/pinot-admin.sh LaunchDataIngestionJob \
  -jobSpecFile /config/job-spec.yml

And the Pinot documentation says that I need to pass in AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. I initially came across a StackOverflow question that described a bunch of approaches that didn’t work for me, but luckily I found a tutorial that showed how to do it.

Effectively we can set these environment variables via the -e parameter, as shown below:

docker exec \
  -e AWS_ACCESS_KEY_ID=<accessKeyId> \
  -e AWS_SECRET_ACCESS_KEY=<secretAccessKey> \
  -it pinot-controller bin/pinot-admin.sh LaunchDataIngestionJob \
  -jobSpecFile /config/job-spec.yml

It’s that simple!

About the author

I'm currently working on short form content at ClickHouse. I publish short 5 minute videos showing how to solve data problems on YouTube @LearnDataWithMark. I previously worked on graph analytics at Neo4j, where I also co-authored the O'Reilly Graph Algorithms Book with Amy Hodler.