Packaging Clojure for Production
Update 2022-05-04: rewrote parts contrasting shadow-cljs and Figwheel Main.
IntroductionLink to Introduction
This post is a look at the various ways of packaging a full-stack Clojure web application for production.
A full-stack Clojure web application has a backend (that is, a HTTP API) written in Clojure, and a frontend (that is, something that runs in your browser and communicates with the backend) written in Clojurescript.
Many posts & templates (on Learn ClojureScript, on this blog and on other blogs) cover setting up a new full-stack Clojure project for local development. There is less talk about how to run one in production. General guidelines are set by classic sources like The 12-Factor App, but more concrete advice is needed as well.
Back in the day the only option for full-stack tooling was Leiningen plus lein-figwheel. These days one can choose between shadow-cljs and Figwheel Main for frontend tooling and between Leiningen and tools.deps aka deps.edn aka Clojure CLI for a build system. Also, for some people, the world has changed from running uberjars on virtual machines to running docker containers on cloud providers.
This post comes with a companion repository on GitHub that contains working examples and documentation for them.
Big Question: Running The BackendLink to Big Question: Running The Backend
The main question you need to answer is:
How do you want to run your backend?
We'll cover three answers to this question: Uberjars, Docker and Git Checkouts. We won't be covering serverless/FaaS backends, which are an interesting topic of their own.
UberjarLink to Uberjar
An Uberjar (also sometimes called a Fat Jar) is a single file that contains a JVM application and all of its dependencies. Uberjars have been the standard way of deploying JVM applications for ages.
To run an Uberjar in production, you need some additional infrastructure for handling starting, restarting, monitoring and logging. Systemd is a good choice for these on Linux, but we won't cover it here.
- A single self-contained runnable file
- Battle-tested standard for running JVM apps
- Can be run anywhere with Java
- Need to set up a Linux Virtual Machine
DockerLink to Docker
(I'll just assume you know what Docker is. If you don't, perhaps Google it.)
If you squint a bit, a Docker container is the same thing as an Uberjar: a single artifact that contains an application and all of its runtime dependencies. However unlike Uberjars, Docker containers aren't restricted to the JVM, but can contain arbitrary native code.
Just like with Uberjars, you probably want some additional
infrastructure to run your Docker containers, instead of just using
docker run manually. There are lots of options for this like
self-hosted Kubernetes, GCP on Google Cloud, EKS on AWS, etc. We won't
go into these here.
We'll cover two different ways of packaging Clojure applications as Docker containers: wrapping an Uberjar into a Docker container image, and constructing a container image directly.
- Can leverage existing Docker infrastructure (e.g. existing Kubernetes cluster)
- Language-agnostic: good for multi-language environments
- Need to set up Docker infrastructure
- Can be harder to debug/profile
Git CheckoutLink to Git Checkout
The most straightforward way of running your application in production is just using a git repository like you would for development. This can be a great way to get started running a prototype, but might not be the best for large-scale use.
- Simple to set up
- Can hack on code directly in production
- Not as robust
- More work to keep multiple deployments in sync
- No single deployable artifact
- Can hack on code directly in production
Big Question: Serving The FrontendLink to Big Question: Serving The Frontend
A secondary question you need to consider is:
How do you want to serve your frontend? (That is, the .html and .js files.)
What matters here is whether you choose to serve your frontend from the backend, or externally
From The BackendLink to From The Backend
Serving the frontend files from the backend simplifies deployment: you
only need to deploy the backend, and the frontend comes for free. In
practice this means you need to bundle your
myapp.js files into the backend, and serve them along the backend
- Backend&frontend versions stay in sync
- Easy deployments
- More complex build step
ExternallyLink to Externally
Instead of bundling your frontend with your backend, you can have a separate solution for serving the frontend files, e.g. a traditional web server like Nginx or a CDN like Cloudflare. This effectively separates your frontend and backend builds and deployments, and might suit projects that have the frontend and backend in separate repositories. However, this might cause a bit of friction with a Clojure/Clojurescript full-stack app that wants to share code between the frontend and backend.
- Simpler build
- Can update backend & frontend separately
- CDNs can speed up delivering the frontend
- Potentially makes frontend/backend code sharing more difficult
- Need to configure/set up the web server or CDN
Small Question: Which Dependency Resolver?Link to Small Question: Which Dependency Resolver?
Before we dive into examples, we need to cover two smaller questions about tooling.
To be able to run or build your web application, you need to fetch all the libraries you're using. The two main solutions for this for Clojure are the traditional Leiningen and the newer deps.edn aka tools.deps aka Clojure CLI.
The main difference between these is that Leiningen does lots of other things, while deps.edn is focused on just fetching the dependencies. While you can use Clojure CLI as a method for running various tools like test runners and uberjar packaging, these things are more tightly built into Leiningen.
My personal opinion is that simple project setups are better off with Leiningen, while more complex ones (a monorepo, multiple microservice, etc.) might benefit from the flexibility of deps.edn. However be prepared for some sharp edges and having to write custom code when dealing with deps.edn.
Small Question: Which Clojurescript Compiler?Link to Small Question: Which Clojurescript Compiler?
Another piece of the full-stack puzzle is compiling your Clojurescript
further complicated by the different needs of local development and
production. When developing the app on your laptop, you probably
want a watcher that recompiles changed
.cljs files and hot
the changes into your browser so that you can see your changes
immediately. For production, you probably want a single
.js file that can be sent to the browser once
and then cached.
Just like with dependency resolvers, you have a couple of options to pick from.
Figwheel (or rather, the tool now called lein-figwheel) was the pioneer of hot code reloading. Shadow-cljs showed up a bit later as the pioneer of using npm deps. Figwheel Main followed and is on par with shadow-cljs features these days.
Both Figwheel and shadow-cljs are compatible with Leiningen and deps.edn, so you can mix and match as you will. For advanced use there are some differences, but they won't matter for this post. Both tools are improving all the time, so check the latest guides when deciding which to use.
We mainly use shadow-cljs at Metosin.
Worked examplesLink to Worked examples
Now that we've talked about what your options are, here are some examples of how the pieces go together.
Uberjar, Leiningen, FigwheelLink to Uberjar, Leiningen, Figwheel
lein uberjar command is a tried-and-true way of building
uberjars. You need an
:uberjar profile in your
you're good to go.
The only possibly tricky bit is including your built frontend code in
the backend if you chose to serve your frontend from the backend. This
is usually best accomplished using leiningen
You can find an example using Leiningen & Figwheel-main in
lein/ directory in the companion repo.
Check the README and the comments in the
project.clj file for more information.
Uberjar, deps.edn, shadow-cljsLink to Uberjar, deps.edn, shadow-cljs
There are a number of ways to build uberjars for deps.edn projects. These include
- tools.build – the new official solution
- depstar – an older solution
- and probably many others
You can find an example using tools.build and shadow-cljs in
deps-uberjar/ directory in the companion repo.
Again, check the README and the comments in the files for more info.
Docker ContainersLink to Docker Containers
If you're running a Docker container, you have one more decision to make:
Will your docker container contain an uberjar?
Some arguments for including an uberjar:
- if you already have a working uberjar build, it's easier
- your Dockerfile is simpler
- you can use a very simple java base image
Some arguments against including an uberjar:
- can't take advantage of docker layer caching
- need a working uberjar build
- more complex images needed, especially if you build your frontend inside your Dockerfile
With an UberjarLink to With an Uberjar
A Dockerfile that wraps an uberjar is pretty simple:
RUN mkdir /app
COPY path/to/project.jar /app
ENTRYPOINT ["java", "-jar", "/app/project.jar"]
If you want, you can also build your uberjar inside your Dockerfile.
Just add a layer that runs
lein uberjar, or use a multi-stage
Dockerfile for smaller output images. Depending on your setup, moving
the uberjar building inside the Dockerfile might not be worth the
trouble though. Uberjar building is very reproducible already on its
Without an UberjarLink to Without an Uberjar
A Dockerfile without an uberjar looks something like the following.
Deps are downloaded in a separate
RUN command to make use of layer
caching. We're using deps.edn here but similar things are possible using Leiningen.
# Base image that includes the Clojure CLI tools
RUN mkdir -p /app
# Prepare deps
COPY deps.edn /app
RUN clojure -P
# Add sources
COPY . /app
CMD clojure -M -m my-project.main
More complexity is added by building the frontend. You can find full
examples of both backend-only and backend-plus-frontend Dockerfiles in
deps-docker/ directory of the companion repo.
Running from a Git CheckoutLink to Running from a Git Checkout
Running from a git checkout is certainly simple. You probably won't
need anything outside your normal dev setup. Here's an example
run.sh that fetches the latest
master, builds the frontend, and
runs your project using clojure CLI.
git reset --hard origin/master
shadow-cljs release app
clojure -M -m my-project.main
For leiningen, the equivalent would be something like:
git reset --hard origin/master
lein cljsbuild once min
Other ConcernsLink to Other Concerns
It's almost time for a summary, but here are some additional concerns that this post doesn't really address.
- Including git revision info in build artefacts
A significant topic we do need to talk about is Ahead-of-Time Compilation.
AOT (Ahead-of-Time) CompilationLink to AOT (Ahead-of-Time) Compilation
Clojure sources can be compiled into Java class files and these class files can be bundled into the uberjar instead of clojure sources. This allows for slightly faster startup time, or might be needed for certain kinds of Java interop where you need to generate an actual Java class from clojure.
However, AOT is not necessary for packaging a clojure app, so we've left it out to simplify the examples. We recommend not doing AOT unless you absolutely have to: it keeps everything simpler at the cost of a slightly longer startup time.
One minor advantage to AOT compilation is that you can specify a custom main class for your uberjar. Then you can run your app with
java -jar my-project.jar
instead of this more verbose command our examples use:
java -cp my-project.jar clojure.main -m my-project.main
It's up to you if this is worth the hassle of AOT compilation though.
PostscriptLink to Postscript
I hope this guide helps you pick a method of building & deploying your full-stack Clojure app that fits your organization, team and process. There are no right answers, only a (sometimes bewildering) amount of different options.