Custom Distributed Task Execution on Github Actions
Using Nx Agents is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. With an enterprise license, you can set up distributed task execution on your own CI provider using the recipe below.
Run Custom Agents on GitHub
Our reusable GitHub workflow represents a good set of defaults that works for a large number of our users. However, reusable GitHub workflows come with their limitations.
If the reusable workflow above doesn't satisfy your needs you should create a custom workflow. If you were to rewrite the reusable workflow yourself, it would look something like this:
.github/workflows/ci.yml
1name: CI
2on:
3 push:
4 branches:
5 - main
6 pull_request:
7
8# Needed for nx-set-shas when run on the main branch
9permissions:
10 actions: read
11 contents: read
12
13env:
14 NX_CLOUD_DISTRIBUTED_EXECUTION: true # this enables DTE
15 NX_CLOUD_DISTRIBUTED_EXECUTION_AGENT_COUNT: 3 # expected number of agents
16 NX_BRANCH: ${{ github.event.number || github.ref_name }}
17 NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
18 NPM_TOKEN: ${{ secrets.NPM_TOKEN }} # this is needed if our pipeline publishes to npm
19
20jobs:
21 main:
22 name: Nx Cloud - Main Job
23 runs-on: ubuntu-latest
24 steps:
25 - uses: actions/checkout@v4
26 name: Checkout [Pull Request]
27 if: ${{ github.event_name == 'pull_request' }}
28 with:
29 # By default, PRs will be checked-out based on the Merge Commit, but we want the actual branch HEAD.
30 ref: ${{ github.event.pull_request.head.sha }}
31 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
32 fetch-depth: 0
33
34 - uses: actions/checkout@v4
35 name: Checkout [Default Branch]
36 if: ${{ github.event_name != 'pull_request' }}
37 with:
38 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
39 fetch-depth: 0
40
41 # Set node/npm/yarn versions using volta
42 - uses: volta-cli/action@v4
43 with:
44 package-json-path: '${{ github.workspace }}/package.json'
45
46 - name: Use the package manager cache if available
47 uses: actions/setup-node@v3
48 with:
49 node-version: 20
50 cache: 'npm'
51
52 - name: Install dependencies
53 run: npm ci
54
55 - name: Check out the default branch
56 run: git branch --track main origin/main
57
58 - name: Initialize the Nx Cloud distributed CI run and stop agents when the build tasks are done
59 run: npx nx-cloud start-ci-run --distribute-on="manual" --stop-agents-after=e2e-ci
60
61 - name: Run commands in parallel
62 run: |
63 # initialize an array to store process IDs (PIDs)
64 pids=()
65
66 # function to run commands and store the PID
67 function run_command() {
68 local command=$1
69 $command & # run the command in the background
70 pids+=($!) # store the PID of the background process
71 }
72
73 # list of commands to be run on main has env flag NX_CLOUD_DISTRIBUTED_EXECUTION set to false
74 run_command "NX_CLOUD_DISTRIBUTED_EXECUTION=false npx nx-cloud record -- nx format:check"
75
76 # list of commands to be run on agents
77 run_command "npx nx affected -t lint,test,build,e2e-ci --parallel=3"
78
79 # wait for all background processes to finish
80 for pid in ${pids[*]}; do
81 if ! wait $pid; then
82 exit 1 # exit with an error status if any process fails
83 fi
84 done
85
86 exit 0 # exits with success status if a all processes complete successfully
87
88 agents:
89 name: Agent ${{ matrix.agent }}
90 runs-on: ubuntu-latest
91 strategy:
92 matrix:
93 # Add more agents here as your repository expands
94 agent: [1, 2, 3]
95 steps:
96 - name: Checkout
97 uses: actions/checkout@v4
98
99 # Set node/npm/yarn versions using volta
100 - uses: volta-cli/action@v4
101 with:
102 package-json-path: '${{ github.workspace }}/package.json'
103
104 - name: Use the package manager cache if available
105 uses: actions/setup-node@v3
106 with:
107 node-version: 20
108 cache: 'npm'
109
110 - name: Install dependencies
111 run: npm ci
112
113 - name: Start Nx Agent ${{ matrix.agent }}
114 run: npx nx-cloud start-agent
115 env:
116 NX_AGENT_NAME: ${{ matrix.agent }}
117There are comments throughout the workflow to help you understand what is happening in each section.