Introduction ¶

Our OneHub chat service is taking shape in our gRPC Series. In Part 5 we introduced Docker for packaging all the components of our services. At that point it only contained the Database (postgres and pgadmin) but we left out the main service binary out of this packaging (via docker-compose.yml file). This was because go binaries needed a rebuild each time source files changed and this in turn would need a rebuild of our docker image - a time consuming process. A live reload/compilation of the service binary without a rebuild of the docker image was missing.

In this post we will rectify this by using the popular Air framework to enable live reloading/compilation of our service binaries.

To summarize, our goals are:

Enable live reloading so that we can avoid restarts (go run cmd/server.go) when making changes.
Ensure our servers can be connected via debugger (eg VSCode)
Package and run the server as a docker image in our docker-compose setup so that we can run a single docker compose up command to bring up everything instead of starting the server seperately.

Getting started ¶

Our code for this part can be found in the PART7_AIR branch of the OneHub repo.

We will be installing 2 main tools:

Air - For enabling live reloading
Delve - For debugging go binaries/services.

Both of these have excellent documentation and we will be pretty much following those here.

To install these:

1go install github.com/go-delve/delve/cmd/dlv@latest
2go install github.com/cosmtrek/air@latest

Initializing and Running Air ¶

Initialize Air by running:

air init

This would create a .air.toml config file that instructs air on how to build binaries, how to run the built binaries (in case any flags/env vars are to be passed) and which files to watch for changes.

Key things to notice are:

root - The root of the project to be watched.
build.cmd - Specifies How to build the binary that we will run (cmd/server.go)
build.bin - How to run the built binary. Instead of just running the binary, we are running it through delve so that we can connec to this via a debugger (such as VSCode). This is especially useful when our binary will be running as a docker image in our docker-compose setup.
build.exclude_dir - Specifies which files/folders are NOT to be watched. This will prevent excessive rebuilding and restarting of our binary.
build.exclude_regex - Specifies which file patterns are to be ignored.

Other than this by default all files and sub directories under the “root” path will be watched.

Running our watched Server ¶

Now we can run our server in watched mode (make sure the database is also running first):

air -c .air.toml

(-c is optional and by default air uses .air.toml as its config file)

With this Air will first build the binary (./tmp/main) and execute it. Try changing one of the source files and see air rebuild and restart the binary! (Note - simply touching a file will not work as air compares file contents too for changes to count).

Dockerizing our setup ¶

Now that we have air setup and live reloading enabled, we are ready to dockerize our services so we can bring up all the components (the database, service binary etc) as one packaged unit (via docker compose up) instead of seperately.

We will create a Dockerfile that will describe how our docker images (for our service) are to be built. We wont go into details on Dockerfiles here. The Dockerfile reference is an amazing resource.

We now have a base image that:

Installs Air, Delve (to be used soon)
Copies our go.mod file and downloads dependencies so that when air takes over it can find these for rebuilding our binary.

Now that we have a base image, we need to add an extra service to our docker-compose.yml:

 1  onehub:
 2    build:
 3      context: .
 4      dockerfile: ./Dockerfile
 5    depends_on:
 6      postgres:
 7          condition: service_healthy
 8            # webapp: condition: service_healthy
 9    volumes:
10      - ./cmd:/app/cmd
11      - ./gen:/app/gen
12      - ./datastore:/app/datastore
13      - ./services:/app/services
14    environment:
15      POSTGRES_DB: ${POSTGRES_DB}
16      POSTGRES_USER: ${POSTGRES_USER}
17      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
18      ONEHUB_DB_ENDPOINT: ${ONEHUB_DB_ENDOINT}
19    ports:
20      - 8080:8080
21      - 9000:9000
22      - 9091:9091

This ensures that all our “source” folders (cmd, gen, datastore, services) are mounted into the docker image as volumes. This is needed so that Air can find the needed files when it initiates a rebuild. We are also exposing an extra port - 9091 - which will be used by Delve to enable remote debugging described later.

Now with the image built we are ready to try out this single endpoint:

1# Bring down if already up
2docker compose down
3
4# Bring it up again
5docker compose up

Now you will see both the database and the service running! Go ahead and make some calls to it either via our CLI or using curl, eg:

oh topics list --username auser --password auser123

You should see results from the service!

A few reminders:

If you add/remove dependencies to go.mod a rebuild of the docker image will be needed:

docker compose build --no-cache

If you add new folders for sources - then make sure to add them to the volumes list in the onhub service section of the docker-compose.yml file (so it is available for builds) and restart (docker compose).
Note that we are adding the gen folder as a Volume mapping. So if the protos change we will simply regenerate the artifacts (buf generate) and restart (docker compose down/up).

Enable Debugging ¶

Bringing up our service from an IDE (say VSCode) was simple and made interactive debugging a breeze. However doing so for a binary running inside a docker container needs a few small changes. Delve is the standard tool for remote attaching/debugging of Go applications. Thankfully we had already installed Delve in our dependencies above.

All that is needed now is to modify our .air.toml file to run our app “wrapped” by Delve instead of running it directly. Our “build.bin” value is now:

  bin = "/go/bin/dlv --listen=:9091 --headless=true --log=true --accept-multiclient --api-version=2 exec --continue ./tmp/main"

Here we are instructing Air to use Delve as the binary which in turns loads our real binary (./tmp/main). Delve is also instructed to start the debugger service on port 9091. (Go) IDEs can now connect to this port and interactively debug the binary.

As an example, if you are using VSCode, add the following configuration to .vscode/launch.json (along with other configurations):

1{
2    "name": "Remote Debugger for OneHub",
3    "type": "go",
4    "request": "attach",
5    "mode": "remote",
6    "port": 9091,
7    "showLog": true,
8    "host": "127.0.0.1"
9}

Now launch this configuration and you will be able to interactively debug the service (eg by setting breakpoints, stepping in/out/over etc).

Conclusion ¶

That’s it we now have a Dockerized setup with Live Reloading and Debugging enabled which lets us:

Package all components and start/stop them as a single unit instead of with multiple start/stop commands
Live reload/rebuild/restart our service when any source files change so we do not have to remember to do this each time.
Enabled a remote debugger so that we can attach a debugger to our service running in a docker container.

We can now build upon these foundations in future posts as we add more features and operational capabilities to our OneHub service.