How to Authorize Your CI/CD to Checkout Additional Repository

Aug 12, 2020

In many cases, you need the content of another private repository to test the current repository. For example, your main repository is a public repo, but you need a secret config file from a private repository, how do you securely setup your CI/CD environment?

Our Goal:

• Your main repo’s CI/CD environment should be able to check out the private repo.

• Anyone who has access to the public repo should not able to access anything from the private repo

In this article, I use CircleCI for example.

Solution1. (Recommended)

Create a separate GitHub account as a Machine User, add this Machine User as a collaborator to the private repo you need to access to, and use the Machine User’s personal access token to check out the private repo in your CI/CD system.

• Sign up a Machine User Github account, let’s say, username = MyORG-Machine

• Create a personal OAuth access token (Account Settings → Developer Settings) with full access of everything in MyORG-Machine

• Add MyORG-Machine as a collaborator to the private, and only grant MyORG-Machine read-only access to that repo.

Code:

# check out your main repo
$ git clone "https://github.com/MyOrg/main-project.git"
# check out your private repo
$ git clone "https://${TOKEN}@github.com/MyOrg/main-project.git"

Pro:

• Even the OAuth access token has full access to MyORG-Machine, but MyORG-Machine‘s access to MyORG is limited. In other words, the token can’t access any other repository in MyORG.

• You can disable personal OAuth access token anytime if you doubt that the token is leaked.

Con:

• You can’t use the same Machine User for other projects that needs access to a different private repo. Otherwise, if the token used in the first project is leaked, then the hacker can also access the private repo in the second project. You need many Machine User for many projects.

Reference:

• Machine User: https://developer.github.com/v3/guides/managing-deploy-keys/#machine-users

• Enable Your Project to Check Out Additional Private Repositories: https://circleci.com/docs/2.0/gh-bb-integration/#enable-your-project-to-check-out-additional-private-repositories

Solution2. (Easy but Risky)

You can grant CI/CD GitHub User Key of your GitHub account (in CircleCI, it is at Project Settings → Checkout SSH Keys → Authorize with GitHub). In other words, you grant your CI/CD system equivalent power as your GitHub Account.

Pro:

• Convenient and fast.

Con:

• If your CI/CD environment been hacked, then the hacker can do everything you can do.

Reference:

• Share User Key with CircleCI: https://circleci.com/docs/2.0/gh-bb-integration/#security

Solution3. (Secure, but too much work)

• manually create an ssh key pair, paste the public key to GitHub Menu → Repository Setting → Deploy Key → Add new Key.

• use a hacky way to include the private key into CI/CD system, put it at $HOME/.ssh.id_rsa_deploy_key or specify that key for git clone command.

Basically, it creates a point-to-point authentication from the machine having your private key to the repository. But securely store, and transfer the private key becomes another difficult problem. And it takes additional effort to manage those key pairs.

Pro:

• Per repo / per ssh level of grained access.

Con:

• Too much work to set up everything, especially managing and injecting the private key.

Reference:

• Generate a new SSH key and adding to your SSH agent: https://help.github.com/en/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent#generating-a-new-ssh-key

• How to specify the private SSH-key to use when executing shell command on Git?: https://stackoverflow.com/questions/4565700/how-to-specify-the-private-ssh-key-to-use-when-executing-shell-command-on-git
  

__________
Learn more:

• Streamlining App Development: 7 Steps to Building the Best CI/CD Pipeline

• App Modernization

• Cloud Transformation

Asim Iqbal

Asim is Enquizit’s CTO and a member of the founding team. He has been an SME on security, storage, and resilience as well as Enquizit’s Lead Architect and VP of Solution Architecture. Among his professional endeavors is the implementation of a weather modeling HPC setup for Environment Canada, storage design and implementation for the Canadian National Institute for the Blind’s library, Media ingestion, encryption, and transcoding architecture for Bell Satellite TV, Cloud infrastructure, resilience, and security architecture and implementation for The Common Application and complete migration of Harvard Business Review’s Primary and DR data center to AWS. He maintains a strong personal interest in frictionless technical designs focused on end-user happiness and employee satisfaction, still thinks that ‘Data Availability Architect’ (from his early days working with HPE) is the coolest certification title ever and is an ex-CISSP. He neither confirms nor denies his purported afflictions with coffee, slow travel, and cats with unbridled spirits.

---