I recently built a few things to make it easier for my team to distribute the tools they’ve been building to the entire engineering organization. I had the advantage of knowing that everybody was on Mac laptops running very close to the latest release. I had some idea of their general habits and blind spots after 18 months of observation. There were a handful of problems I needed to solve here, and a private tap looked like it would solve all of them.

The problems were:

Discoverability. We can document things all we want using Notion or whatever the corporate information system of the moment is, but unless somebody thinks to go hunting, they won’t find the documentation. brew search our-github-org is easy to remember, however, because everybody is already using homebrew.

Automated installation. It’s straightforward to install the latest GitHub release of something in a shell script, if you can get the person to run the shell script. Running brew install foo is even easier. It’s also easy to say this out loud to somebody if you can’t type it.

Updates. We can mostly coax people into onboarding into new repos by making a just setup recipe conventional in as many of them as we touch, but only mostly. People definitely don’t remember to run setup scripts again to update. brew upgrade, on the other hand, is something they will run occasionally.

Signed Mac executables. We’re building some tools that are notarized Mac executables with installers. Brew knows how to install all the things.

The obstacles to using a private tap are mostly that distribution of not-open-source software is of zero interest to the Hombrew project; they’re a package manager for open source for MacOS (and Linux too). You have to build this yourself. Fortunately, it’s not hard.

By Fediverse friend request, I share my approach. Here’s the outline:

• Use a download strategy that can fetch assets from release in private repos.
• Publish internal tools to your tap with formulas that mark them as using this new strategy.
• Show people the slight bit of magic needed to tap a private repo.

You can do an Internet search for articles about how to write a download strategy that uses curl. I chose to write a download strategy that uses gh, the GitHub cli. I’ve been using gh in actions for a while because it’s so easy to use for certain tasks, like, well, downloading artifacts from private repos.

If you’ve maintained your own tap before, skip down to the strategy section and snag that. Read on for the full details if making brew taps is new to you.

Get gh installed

Get everybody set up with gh, because they’ll need it installed to use this strategy. Using gh as a credentials helper for GitHub makes the next step more convenient, but is not required. GitHub has good installation instructions. You can script it with some human action needed like this:

brew install gh
gh auth login
gh auth setup-git # optionally

The tap repo

Now get your tap repo set up.

• Create a private repo in your organization named your-org/homebrew-tap. You can leave out the “homebrew” part of the name in most of the usage instructions, because it is implied. Naming the repo unambiguously is merely a convention.

• Make sure everybody who needs to install tools from it has at least read access. Probably you have a GitHub team that’s “all developers” or the equivalent.

• Make sure everybody who needs to publish tools by hand to it has at least write access.

• Create a fine-grained GitHub token that can clone the repo and write new commits to it. Make this available as an organization secret, so any repo can use it in a release workflow.

• Finally, get everybody to tap it, using one of these invocations:

brew tap your-org/tap git://github.com/your-org/homebrew-tap.git
# to use ssh to access it
brew tap your-org/tap https://github.com/your-org/homebrew-tap
# or if you're authing with the gh helper:
brew tap your-org/tap

Now we have everybody tapping an empty cask. Let’s fill it.

Formula files

These are predictable. Generating them is a template rendering problem that can be solved in a number of ways. Here’s a template I use for typical formulas. You can write some yourself by looking at examples.

Generating them by hand is, however, de trop. This is what computers are for, and especially what workflows that generate GitHub releases are for.

You can find actions to do this by searching on GitHub. Here’s where you’ll want the API token you generated above with the ability to make commits to the tap repo. You’ll do a set of builds in a workflow, create the release, upload assets, use whatever you’ve chosen to generate a formula file, then finally commit the formula file to your tap repo.

Here’s an example tap update step from one of my repos. Note the two access tokens passed in as env vars. One is that token that can commit to your tap repo; this needs to be provided to gh via the env var GH_TOKEN. The other is the token GitHub generates for the workflow run, which has access to the tool repo. You’ll want to to read release information from the GitHub api with gh, which I found a million times easier to do than clunking my way through workflow step inputs and outputs. (Your mileage may vary.)

In fact, you can probably use the gh build-in gotemplate formatting feature to make it emit a formula file, if you can cope with really long inline template strings.

gh release view -R org/repo --template "some long template string here"

The twist here is that we want to change up the standard template by adding a custom download strategy to our tap.

Downloading release assets (by strategy)

Here’s the download strategy itself. You can either embed this into each formula (the lazy way, which I chose), or put it into a file in your tap repo and then require that file in each formula.

require "download_strategy"
require "utils/formatter"
require "utils/github"
require "system_command"

class GitHubCliDownloadStrategy < CurlDownloadStrategy
  require "utils/formatter"
  require "utils/github"
  require "system_command"

  def initialize(url, name, version, **meta)
    super
    # Extract owner and repo from the URL
    match_data = %r{^https?://github\.com/(?<owner>[^/]+)/(?<repo>[^/]+)/releases/download}.match(@url)
      return unless match_data

      @owner = match_data[:owner]
      @repo = match_data[:repo]
      @filename = File.basename(@url)
  end

  def fetch(timeout: nil)
    ohai "Downloading #{url} using GitHub CLI"
    if cached_location.exist?
        puts "Already downloaded: #{cached_location}"
    else
      begin
          temporary_path.dirname.mkpath

          # note path hack
          system_command("/opt/homebrew/bin/gh", args: [
                "release", "download",
                "-R", "#{@owner}/#{@repo}",
                "--pattern", "#{@filename}",
                "-D", "#{temporary_path}"
             ], print_stderr: true)
      rescue ErrorDuringExecution
          raise GitHubCliDownloadStrategyError, "GitHub CLI download failed for: #{url}"
      end
      cached_location.dirname.mkpath

      # Find the downloaded file in the temporary path
      downloaded_file = Dir["#{temporary_path}/*"].first

      if downloaded_file
          FileUtils.mv(downloaded_file, cached_location)
      else
          raise GitHubCliDownloadStrategyError, "Downloaded file not found in #{temporary_path}"
      end
    end

    symlink_location.dirname.mkpath
    FileUtils.ln_s cached_location.relative_path_from(symlink_location.dirname), symlink_location, force: true
  end
end

As you can see from the horrible hack noted in the comment, I didn’t bother figuring out how paths are set up for homebrew scripts. Maybe you know more than I do.

You’ll want to edit the part of your formula file where you list available asset files by OS and architecture to mention the strategy like this:

if OS.mac? && Hardware::CPU.arm?
    url    "https://github.com/ceejbot/codefact/releases/download/v1.0.2/codefact-aarch64-apple-darwin.tar.gz", using: GitHubCliDownloadStrategy
    sha256 "0e0d03a2f787f6d875ff02ce91cf495cc95878ace96b9a9c8f3073a6a9688b44"
end

Test it

Get some people to test the whole process and make sure everything works. Verify that your docs are clear enough that even the people who break everything can manage to make it work. Test your release workflows. Test updates.

You’re done.

Autogenerating formula files (Rust bins only)

As I advised you to do above, I looked for GitHub actions to generate formula files for me from a template. There’s one that looks heavily used, but I immediately had difficulties with python’s requests module with it. My patience for debugging python packaging problems is about zero these days, so I banged out a Rust cli tool to write exactly the formula file that the late cargo-dist used to write for me. This tool, formulaic, also has a flag for specifying using the gh strategy instead of figuring it out automatically.

This tool reads the latest GitHub release and the Cargo manifest file it’s given the path to, which means it only works for Rust executables. It’s a quite predictable thing that fills out a template. The source is on GitHub. You can see its self-generated formula file in my own Homebrew tap. Snag and edit to taste if it saves you some time, so long as you also share your changes. (See the license.)

Random shasum trivia

GitHub seems to have recently started adding sha 256 sums to release asset data, so they don’t have to be calculated the way I’m doing it. Unless you’re very cautious, that is. You can use gh to get the shasum of any specific asset:

gh release view -R ceejbot/tomato --json assets | jq -r '.assets[] | .name, .digest'

Here’s something I learned while observing that for some reason there are three different ways to get a sha256 sum of a file out of the box on MacOS. There are two output variations, because of course there are.

Note that Homebrew wants the bare hex string, without filename decoration of any kind. If you use the github-generated shasum, you’ll need to trim the sha256: prefix.