End to End CI/CD pipeline using GitHub Actions for Android Application

In this article, you will get a brief idea about how to create an End to End CI/CD Pipeline using GitHub Actions for an Android Application.

Project Source Code : LINK

Here, we will be covering some use cases like how to trigger one workflow from another workflow, how to run 2 jobs which depends upon each other, how to add public ip of GitHub Actions in security groups of Jfrog which is running in an EC2 , on port 8082 — so that GitHub Actions can access Jfrog to upload the .apk file into repository, how to integrate SonarQube , Teams with GitHub Actions , how to create a cron job in workflow, how to delete artifacts which are created during the workflow in GitHub Actions, how to clean caches in GitHub Actions which gets created every time your run a workflow.

Step by Step Process :

To create a workflow, go to Actions in your GitHub Repository and choose a template yml file or click “set up a workflow yourself” — for me I have chosen Android CI as my application is android application.

This basic template you will get to do CI part of Android Application.

Let me explain you some of the terms used in workflow file:

name : The name of the workflow as it will appear in the “Actions” tab of the GitHub repository. Like here it is “Android CI”

on: Specifies the trigger for this workflow. So here the workflow will be triggered when there is a push event in “main” branch and pull_request event in “main” branch

jobs: A workflow job is a set of steps that execute on the same runner. We can have multiple jobs in a single workflow yml file. Groups together all the jobs that run in the Android CI workflow. Here, in the example there is a single job whose name is build

runs-on: Configures the job to run on the latest version of an Ubuntu Linux runner. This means that the job will execute on a fresh virtual machine hosted by GitHub. You can use windows and macOS runner too.

steps: Groups together all the steps that run in the build job. Each item nested under this section is a separate action or shell script.

uses: actions/checkout@v3 : The uses keyword specifies that this step will run v3 of the actions/checkout action. This is an action that checks out your repository onto the runner, allowing you to run scripts or other actions against your code (such as build and test tools). You should use the checkout action any time your workflow will run against the repository’s code.

uses: actions/setup-java@v3 : This step uses the actions/setup-java@v3 action to install the specified version of the JDK (this example uses v11) of distribution: ‘temurin’

run: chmod +x gradlew: The run keyword tells the job to execute a command on the runner. In this case, you are granting execute permission for gradlew

run: ./gradlew build: In this case you are building the code using gradle

After this , Click on Start Commit and add comment and click Commit. This will create a basic android CI workflow in GitHub Action.

To create a secret in GitHub Actions  Go to Settings and then click on Secrets and then to Actions and create different secrets .

Let’s modify the android.yml file : Full Workflow

Let’s discuss part by part

Here, the workflow is getting triggered whenever we push into the branches “main” or “qa” or “develop” and whenever we pull request into the branches “main” or “qa”.

You can use environment variables to store information that you want to reference in your workflow. You reference environment variables within a workflow step or an action, and the variables are interpolated on the runner machine that runs your workflow. Commands that run in actions or workflow steps can create, read, and modify environment variables.

You can define environment variables that are scoped for:

Here in the workflow we can have created a env at the top level with variable name as “AWS_DEFAULT_REGION” and assigned value as “ap-south-1”

uses: actions/setup-java@v3 : This step uses the actions/setup-java@v3 action to install the specified version of the JDK (this example uses v11) of distribution: ‘temurin’

run: chmod +x gradlew: The run keyword tells the job to execute a command on the runner. In this case, you are granting execute permission for gradlew

run: ./gradlew clean: Gradle clean will delete a build directory if already present.

run: ./gradlew lint: It will detect poorly structured code that can impact the reliability and efficiency of your Android apps and make your code harder to maintain

run: ./gradlew build: A process of building a Gradle project

run: ./gradlew jacocoTest: The JacocoReport task can be used to generate code coverage reports in different formats

In this step we are integrating GitHub Actions with SonarQube. To make your workflows faster and more efficient, you can create and use caches for dependencies and other commonly reused files.

In the Cache SonarQube Package Step and Cache Gradle Package we are caching SonarQube packages and Gradle Packages with given path where runner stores the cache. The new cache will use the key you provided and contains the files you specify in path and alternative restore key used if no cache hit occurs for key, these restore keys are used sequentially in the order provided to find and restore a cache.

In the next step we are doing code analysis using SonarQube. We have added environment variables whose values are as secrets which you can use in your workflows as environment variables.

The secrets in GitHub Actions are defined as {{ secrets.secret_name }}. Here we have added SonarQube Token and SonarQube URL as secrets. Then we are running the command in runner as “./gradlew sonarqube” and passing the projectkey.

In build.gradle we have added the plugins for SonarQube

We can see that SonarQube analysis is passed and coverage report is greater than 80%

In the next step “Date and Time” , we have evaluated date and time using Linux command and created outputs in the step by writing to stdout in the format of ::set-output name=<name>::<value>. A step can have multiple outputs. Steps that create outputs must have unique ids.

current_date_time::$(date +”%d-%m-%Y-%H-%M-%S”)  Here current_date_time variable is in the format of %d : Day of the month , %m : Month , %Y : Year , %H : Hour, %M : Minutes, %S : Seconds

Here, the output name is “current_date_time” and id of the step of “Date and Time” is “date” which is unique name in the workflow.

To use this parameter in the job we can use in the way {{ steps.<step-id>.outputs.<output-name> }} . Here in the example it is {{ steps.date.outputs.current_date_time }}

In the next step “Copy APK files to a directory” we are creating a directory structure to store the Debug and Release APK files in the format

apk-files > debug > app-debug-11–11–2022–09–09–12–36.apk

apk-files > release> app-release-unsigned-11–11–2022–09–09–12–36.apk

In the last step “Upload apk-files” Directory we are uploading artifactory — APK Files which we created so that in the deploy “job” we can download this artifactory as it will be using a new ubuntu runner. Here {{ github.workspace }} is default path for the checkout action. The path which we want to upload is the apk-files directory and if-no-files is present in the path , then ignore this step.

In this step we are integrating teams with GitHub Actions. This action take your GitHub token and the webhook URL which is generated during the configuration part .

Create a teams channel add people who should be notified for workflow success and fail steps . Click Connectors in the channel and then choose “Incoming Webhook” and add and then configure and then add a name and copy the URL and paste it as a secret in your GitHub Secrets and use it in your workflow.

If you see the last part, we are creating an outputs variable name as “CURRENT_DATE_TIME” and passing the date time value . Since this variable we want to use in another job “deploy” . To pass any variables from 2 different jobs we need to create output values like this.

Then we are creating another job named as “deploy” where we are adding our CD part of the Workflow.

Here “needs” : build means after successfully executing the “build” job only this “deploy” job will run.

Then the if condition is telling that if branches are “qa” or “master” then only this steps will run inside the runner.

Then again in the steps checkout repository is happening and in the next step “Download apk-files Artifactory” , we are downloading the artifact which we just uploaded in “build” job. We have mentioned the path where we need to download the artifact

Then in “Display structure of downloaded files” step we are checking the directory structure of downloaded artifacts.

In the next step “Public IP of GitHub Hosted Runner”  we are generating the Public IP of the GitHub Hosted Runner by using the action haythem/public-ip@1.3

Then in the next step we are adding the Public IP to Security Group in which Jfrog is running in the EC2 Instance so that GitHub Actions can access the Jfrog Page at 8082 using AWS CLI Commands “authorize-security-group-ingress”. For this we need to create a user with having EC2 Full Access permission and give programmatic access to get access_key_id and secret_access_key.

In the first step we are downloading the Jfrog CLI with latest version. Add the JF_URL which is the URL of the artifactory where we are storing the .apk files and access token which we can create in Admin → User Management → Access Token .

Also set the “Password Encryption Policy” to Unsupported for demo purpose.

Then in the next step we are creating folders for QA and Master Branch. Where in script we are using if and else condition , that if GitHub branch is qa then we will create a directory QA in apk-files directory else master. Now the directory structure will be like:

apk-files > qa > debug > app-debug-11–11–2022–09–09–12–36.apk

apk-files > qa> release> app-release-unsigned-11–11–2022–09–09–12–36.apk

apk-files > master > debug > app-debug-11–11–2022–09–09–12–36.apk

apk-files > master > release> app-release-unsigned-11–11–2022–09–09–12–36.apk

In the step “Upload APK files to Jfrog” we are using Jfrog CLI commands to upload apk files from Ubuntu Runner to Jfrog Artifactory –> “android-artifact”

jf rt u — url ${{ secrets.JF_URL }} — user ${{ secrets.JF_USER }} — password ${{ secrets.JF_PASSWORD }} apk-files/qa/debug/app-debug-${{ needs.build.outputs.CURRENT_DATE_TIME }}.apk android-artifact/

Here, u means upload , — url means Artifactory Repository URL (android-artifact) one, — user means Username of Jfrog UI and –password means Password of Jfrog UI , <path of the file want to upload> <artifact-repo-name>/

This URL to file is the JF_URL which we need to add a secret in GitHub Actions.

Then in the next step , we are removing the GitHub Actions Public IP from the security group of Jfrog EC2 Instance . if: always() make sure that this step runs always although if any steps fails.

Then in the last step we are sending notifications to teams. You can see all these information will go to teams.

Use Case : If we want to delete the caches which are formed after running each workflow

Here, we are using workflow_run command  It allows you to execute a workflow based on execution or completion of another workflow. So, here we are telling that “Clear Cache” Workflow will run only when workflow “Android CI and CD” workflow will complete (type of activity) successfully. Then we add permissions as write in a top-level key, to apply to all jobs in the workflow. When you add the permissions key within a specific job, all actions and run commands within that job that use the GITHUB_TOKEN gain the access rights you specify.

After that we run a script where we are first listing all Caches using JavaScript command and then we are deleting caches using their ID.

You can see all the caches got deleted which got created in the previous workflow.

Use Case : If we want to delete the artifacts which are formed when we uploaded the artifacts (apk-files) so that we can pass that directory from build job to deploy job using cron job — Every HOUR

Here, we are running a cron job which will run every hour and delete all artifacts which got created . Here we are passing the GitHub token in purge-artifacts action.

Use Case : How to create Self Hosted Runner and how to configure the self-hosted runner application as a service.