Building a Secret Scanner in Julia: A GitLeaks Alternative

There is a tool for scanning secrets, passwords, and API key leaks called GitLeaks. It’s a very popular project, but in my opinion, its popularity is undeserved. The code quality is questionable, it doesn’t allow specifying custom regex pattern lists for scanning, and it fails to produce proper JSON in stdout, making it useless for automation or backend use. This was a great opportunity to: To use Julia – one of the best programming languages, which is unfairly considered niche. Its applications go far beyond HPC. It’s perfectly suited for solving a wide range of problems. Learn how to properly publish projects on GitHub. Learn how to create Linux packages. We won’t dwell too long on the code itself. Code is better written or read—discussing it isn’t very interesting. The script works as follows: Usage: julia leaquor.jl [options] Options: -h, --help Display this help message. --json Output results in JSON format. --output-file FILE Write JSON results to the specified file. --patterns FILE Load additional patterns from a YAML file. --ignore-files LIST Comma-separated list of files to ignore (e.g., "file1.txt,file2.json"). --repo URL Clone and scan a GitHub repository (e.g., https://github.com/user/repo.git). --dir PATH Scan a specific directory on the file system. --entropy-threshold FLOAT Set custom entropy threshold (default: 3.5). --log-file FILE Write logs to the specified file. Arguments: Either --repo or --dir must be provided. Examples: julia leaquor.jl --repo https://github.com/user/repo.git --json --output-file out.json julia leaquor.jl --dir ./my_project --patterns custom_patterns.yaml | jq . --patterns – A YAML file with custom regex patterns for scanning. --dir – Specifies a directory to scan. --repo – Allows scanning a GitHub repository directly. More interesting is how to properly structure this for GitHub: Docker and DEB Packages In one Dockerfile, we build the project in Julia. Julia is no different from Python or other more popular languages in this regard. In a second Dockerfile, we also build the project but add nfpm a tool that replaces the standard toolchain for DEB and RPM packages. # Generate DEB packages RUN nfpm pkg --config nfpm.yaml --target leaquor.deb # Final stage to output packages FROM scratch COPY --from=packager /app/leaquor.deb /leaquor.deb The last two directives build and copy the package to the current directory. Make A Makefile is created to handle Docker builds: # Build the Docker image docker-build: @echo "Building Docker image $(DOCKER_IMAGE)..." docker build -t $(DOCKER_IMAGE) . # Run the Docker container docker-run: @echo "Running Docker container $(DOCKER_CONTAINER)..." docker run --rm -v $(PWD):/app $(DOCKER_IMAGE) --help If called without arguments, it displays the help menu: make Available targets: test - Run unit tests docker-build - Build the Docker image docker-run - Run the Docker container docker-push - Push the Docker image to Docker Hub docker-clean - Clean up Docker artifacts (images, containers, volumes) docker-clean-all - Force cleanup of all Docker images and artifacts release - Create a GitHub release open-issue - Open a new issue on GitHub create-pr - Create a pull request list-issues - List open issues list-prs - List open pull requests deploy-docs - Deploy documentation to GitHub Pages check-workflows - Check GitHub Actions workflow status generate-changelog - Generate a changelog This command pushes the image to DockerHub: # Push the Docker image to Docker Hub docker-push: docker-build @echo "Logging into Docker Hub..." @docker login -u $(DOCKER_HUB_USERNAME) @echo "Tagging image for Docker Hub..." docker tag $(DOCKER_IMAGE) $(DOCKER_HUB_REPO):latest @echo "Pushing image to Docker Hub..." docker push $(DOCKER_HUB_REPO):latest @echo "Image pushed successfully to $(DOCKER_HUB_REPO):latest" Nothing stops us from using GitHub CLI to tag the repository, upload it, and create a GitHub release (that’s the section on the left with version numbers like v1.0.0): # Target: Create a Git tag and release on GitHub .PHONY: release release: @echo "Creating Git tag and releasing on GitHub..." @read -p "Enter the version number (e.g., v1.0.0): " version; \ git tag -a $$version -m "Release $$version"; \ git push origin $$version; \ gh release create $$version --generate-notes @echo "Release $$version created and pushed to GitHub." # Upload .deb package to GitHub release upload-release: @if [ ! -f "leaquor.deb" ]; then \ echo "Error: leaquor.deb not found. Please run 'make package' first."; \ exit 1; \ fi @echo "Uploading leaquor.deb to GitHub release $(RELEASE_TAG)..." gh release upload $(RELEASE_TAG

Apr 24, 2025 - 16:48

Building a Secret Scanner in Julia: A GitLeaks Alternative

There is a tool for scanning secrets, passwords, and API key leaks called GitLeaks.

It’s a very popular project, but in my opinion, its popularity is undeserved. The code quality is questionable, it doesn’t allow specifying custom regex pattern lists for scanning, and it fails to produce proper JSON in stdout, making it useless for automation or backend use.

This was a great opportunity to:

To use Julia – one of the best programming languages, which is unfairly considered niche. Its applications go far beyond HPC. It’s perfectly suited for solving a wide range of problems.
Learn how to properly publish projects on GitHub.
Learn how to create Linux packages.

We won’t dwell too long on the code itself. Code is better written or read—discussing it isn’t very interesting.

The script works as follows:

Usage: julia leaquor.jl [options]

Options:
  -h, --help          Display this help message.
  --json              Output results in JSON format.
  --output-file FILE  Write JSON results to the specified file.
  --patterns FILE     Load additional patterns from a YAML file.
  --ignore-files LIST Comma-separated list of files to ignore (e.g., "file1.txt,file2.json").
  --repo URL          Clone and scan a GitHub repository (e.g., https://github.com/user/repo.git).
  --dir PATH          Scan a specific directory on the file system.
  --entropy-threshold FLOAT Set custom entropy threshold (default: 3.5).
  --log-file FILE     Write logs to the specified file.

Arguments:
  Either --repo or --dir must be provided.

Examples:
  julia leaquor.jl --repo https://github.com/user/repo.git --json --output-file out.json
  julia leaquor.jl --dir ./my_project --patterns custom_patterns.yaml | jq .

--patterns – A YAML file with custom regex patterns for scanning.
--dir – Specifies a directory to scan.
--repo – Allows scanning a GitHub repository directly.

More interesting is how to properly structure this for GitHub:

Docker and DEB Packages

In one Dockerfile, we build the project in Julia. Julia is no different from Python or other more popular languages in this regard.

In a second Dockerfile, we also build the project but add nfpm a tool that replaces the standard toolchain for DEB and RPM packages.

# Generate DEB packages
RUN nfpm pkg --config nfpm.yaml --target leaquor.deb

# Final stage to output packages
FROM scratch
COPY --from=packager /app/leaquor.deb /leaquor.deb

The last two directives build and copy the package to the current directory.

Make

A Makefile is created to handle Docker builds:

# Build the Docker image
docker-build:
    @echo "Building Docker image $(DOCKER_IMAGE)..."
    docker build -t $(DOCKER_IMAGE) .

# Run the Docker container
docker-run:
    @echo "Running Docker container $(DOCKER_CONTAINER)..."
    docker run --rm -v $(PWD):/app $(DOCKER_IMAGE) --help

If called without arguments, it displays the help menu:

make
Available targets:
  test             - Run unit tests
  docker-build     - Build the Docker image
  docker-run       - Run the Docker container
  docker-push      - Push the Docker image to Docker Hub
  docker-clean     - Clean up Docker artifacts (images, containers, volumes)
  docker-clean-all - Force cleanup of all Docker images and artifacts
  release          - Create a GitHub release
  open-issue       - Open a new issue on GitHub
  create-pr        - Create a pull request
  list-issues      - List open issues
  list-prs         - List open pull requests
  deploy-docs      - Deploy documentation to GitHub Pages
  check-workflows  - Check GitHub Actions workflow status
  generate-changelog - Generate a changelog

This command pushes the image to DockerHub:

# Push the Docker image to Docker Hub
docker-push: docker-build
    @echo "Logging into Docker Hub..."
    @docker login -u $(DOCKER_HUB_USERNAME)
    @echo "Tagging image for Docker Hub..."
    docker tag $(DOCKER_IMAGE) $(DOCKER_HUB_REPO):latest
    @echo "Pushing image to Docker Hub..."
    docker push $(DOCKER_HUB_REPO):latest
    @echo "Image pushed successfully to $(DOCKER_HUB_REPO):latest"

Nothing stops us from using GitHub CLI to tag the repository, upload it, and create a GitHub release (that’s the section on the left with version numbers like v1.0.0):

# Target: Create a Git tag and release on GitHub
.PHONY: release
release:
    @echo "Creating Git tag and releasing on GitHub..."
    @read -p "Enter the version number (e.g., v1.0.0): " version; \
    git tag -a $$version -m "Release $$version"; \
    git push origin $$version; \
    gh release create $$version --generate-notes
    @echo "Release $$version created and pushed to GitHub."

# Upload .deb package to GitHub release
upload-release:
    @if [ ! -f "leaquor.deb" ]; then \
            echo "Error: leaquor.deb not found. Please run 'make package' first."; \
            exit 1; \
        fi
    @echo "Uploading leaquor.deb to GitHub release $(RELEASE_TAG)..."
    gh release upload $(RELEASE_TAG) leaquor.deb --repo $(GITHUB_REPO)
    @echo "Upload complete"

[

Test

In this case, we simply run the script in a Docker container against a "leaky-repo"—a test repository that definitely contains secret leaks.

# Test target to validate the script
test: docker-build clone-test-repo
    @echo "Running tests..."
    docker run --rm \
            -v $(PWD)/$(JULIA_SCRIPT):/app/$(JULIA_SCRIPT) \
            -v $(PWD)/$(TEST_REPO_DIR):/app/$(TEST_REPO_DIR) \
            --name $(DOCKER_CONTAINER) \
            $(DOCKER_IMAGE) \
            julia /app/$(JULIA_SCRIPT) --dir /app/$(TEST_REPO_DIR) \
            --patterns patterns.yaml \
            --output-file results.json \
                --log-file /app/test.log
    @echo "Tests completed. Check test.log for details."

Pipeline

Finally, to keep everything GitOps-compliant, let’s set up a GitHub pipeline:

name: CI/CD Pipeline

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  build-and-test:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v3

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v2

    - name: Log in to Docker Hub
      if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
      uses: docker/login-action@v2
      with:
        username: ${{ secrets.DOCKER_HUB_USERNAME }}
        password: ${{ secrets.DOCKER_HUB_TOKEN }}

    - name: Build Docker image
      run: make docker-build

    - name: Run tests
      run: make test

    - name: Build package (on tag push)
      if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
      run: make package

    - name: Push Docker image to Docker Hub (on tag push)
      if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
      run: |
        docker tag leaquor-image:latest ${{ secrets.DOCKER_HUB_REPO }}:${{ github.ref_name }}
        docker push ${{ secrets.DOCKER_HUB_REPO }}:${{ github.ref_name }}

Here, everything is very simple because we wrapped all logic in Make. We can just run make docker-build, make test, and automate it on push.

In reality, it’s all very straightforward. Julia can be used for standard day-to-day tasks, and Docker and GitHub make it even easier.