Integrating C libraries into Go (Example with libbz2)

Hi, my name is Walid, a backend developer who’s currently learning Go and sharing my journey by writing about it along the way. Resource : The Go Programming Language book by Alan A. A. Donovan & Brian W. Kernighan Matt Holiday go course In this tutorial, we will demonstrate how to integrate Go with the libbzip2 compression library using cgo. We will keep the C code in a separate file and call it from Go to compress a file. Prerequisites Before we begin, ensure you have the following installed: libbzip2 Development Library: On Debian-based systems: sudo apt install libbz2-dev Project Structure To keep things organized, we will separate our Go and C code into different files. Create the following directory structure: project-directory/ ├── main.go ├── bz2compress.c └── bz2compress.h Writing the C Code 1. Create bz2compress.h This header file declares the functions we will implement in bz2compress.c. #ifndef BZ2COMPRESS_H #define BZ2COMPRESS_H #include int bz2compress(bz_stream strm, int action, char in, unsigned int inlen, char out, unsigned int outlen); bz_stream bz2alloc(void); void bz2free(bz_stream* strm); #endif // BZ2COMPRESS_H 2. Create bz2compress.c This file implements the functions declared in the header. #include "bz2compress.h" #include int bz2compress(bz_stream strm, int action, char in, unsigned int inlen, char out, unsigned int outlen) { strm->next_in = in; strm->avail_in = inlen; strm->next_out = out; strm->avail_out = outlen; int ret = BZ2_bzCompress(strm, action); inlen -= strm->avail_in; outlen -= strm->avail_out; strm->next_in = strm->next_out = NULL; return ret; } bz_stream bz2alloc(void) { return (bz_stream)calloc(1, sizeof(bz_stream)); } void bz2free(bz_stream strm) { free(strm); } Writing the Go Code Now, let's write the Go code to use these C functions. 3. Create main.go package main /* #cgo LDFLAGS: -lbz2 #include "bz2compress.h" / import "C" import ( "fmt" "io" "os" "unsafe" ) func main() { inputFile := "input.txt" outputFile := "output.bz2" inFile, err := os.Open(inputFile) if err != nil { fmt.Fprintf(os.Stderr, "Error opening input file: %v\n", err) os.Exit(1) } defer inFile.Close() outFile, err := os.Create(outputFile) if err != nil { fmt.Fprintf(os.Stderr, "Error creating output file: %v\n", err) os.Exit(1) } defer outFile.Close() const bufferSize = 64 1024 inBuffer := make([]byte, bufferSize) outBuffer := make([]byte, bufferSize) strm := C.bz2alloc() defer C.bz2free(strm) if ret := C.BZ2_bzCompressInit(strm, 9, 0, 0); ret != C.BZ_OK { fmt.Fprintf(os.Stderr, "BZ2_bzCompressInit failed: %d\n", ret) os.Exit(1) } defer C.BZ2_bzCompressEnd(strm) for { inLen, err := inFile.Read(inBuffer) if err != nil && err != io.EOF { fmt.Fprintf(os.Stderr, "Error reading input file: %v\n", err) os.Exit(1) } if inLen == 0 { break } inLenC := C.uint(inLen) outLenC := C.uint(bufferSize) ret := C.bz2compress(strm, C.BZ_RUN, (C.char)(unsafe.Pointer(&inBuffer[0])), &inLenC, (C.char)(unsafe.Pointer(&outBuffer[0])), &outLenC) if ret != C.BZ_RUN_OK { fmt.Fprintf(os.Stderr, "BZ2_bzCompress failed: %d\n", ret) os.Exit(1) } if _, err := outFile.Write(outBuffer[:outLenC]); err != nil { fmt.Fprintf(os.Stderr, "Error writing to output file: %v\n", err) os.Exit(1) } } fmt.Println("Compression completed successfully.") } Building and Running the Application Initialize the Go Module go mod init project-directory Build the Project go build Run the Application ./project-directory Ensure input.txt exists before running the program. The compressed output will be saved as output.bz2. Conclusion By following this tutorial, you have successfully integrated C code with Go using cgo. Keeping the C code separate improves maintainability and allows better modularity. This approach can be extended to use other C libraries in Go applications efficiently. If you have any questions or suggestions, feel free to leave a comment below!

Apr 1, 2025 - 07:46

Integrating C libraries into Go (Example with libbz2)

Hi, my name is Walid, a backend developer who’s currently learning Go and sharing my journey by writing about it along the way.
Resource :

The Go Programming Language book by Alan A. A. Donovan & Brian W. Kernighan
Matt Holiday go course

In this tutorial, we will demonstrate how to integrate Go with the libbzip2 compression library using cgo. We will keep the C code in a separate file and call it from Go to compress a file.

Prerequisites

Before we begin, ensure you have the following installed:

libbzip2 Development Library:
- On Debian-based systems:
```
sudo apt install libbz2-dev
```

Project Structure

To keep things organized, we will separate our Go and C code into different files. Create the following directory structure:

project-directory/
├── main.go
├── bz2compress.c
└── bz2compress.h

Writing the C Code

1. Create `bz2compress.h`

This header file declares the functions we will implement in bz2compress.c.

#ifndef BZ2COMPRESS_H
#define BZ2COMPRESS_H

#include 

int bz2compress(bz_stream *strm, int action, char *in, unsigned int *inlen, char *out, unsigned int *outlen);
bz_stream* bz2alloc(void);
void bz2free(bz_stream* strm);

#endif // BZ2COMPRESS_H

2. Create `bz2compress.c`

This file implements the functions declared in the header.

#include "bz2compress.h"
#include 

int bz2compress(bz_stream *strm, int action, char *in, unsigned int *inlen, char *out, unsigned int *outlen) {
    strm->next_in = in;
    strm->avail_in = *inlen;
    strm->next_out = out;
    strm->avail_out = *outlen;
    int ret = BZ2_bzCompress(strm, action);
    *inlen -= strm->avail_in;
    *outlen -= strm->avail_out;
    strm->next_in = strm->next_out = NULL;
    return ret;
}

bz_stream* bz2alloc(void) {
    return (bz_stream*)calloc(1, sizeof(bz_stream));
}

void bz2free(bz_stream* strm) {
    free(strm);
}

Writing the Go Code

Now, let's write the Go code to use these C functions.

3. Create `main.go`

package main

/*
#cgo LDFLAGS: -lbz2
#include "bz2compress.h"
*/
import "C"
import (
    "fmt"
    "io"
    "os"
    "unsafe"
)

func main() {
    inputFile := "input.txt"
    outputFile := "output.bz2"

    inFile, err := os.Open(inputFile)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error opening input file: %v\n", err)
        os.Exit(1)
    }
    defer inFile.Close()

    outFile, err := os.Create(outputFile)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error creating output file: %v\n", err)
        os.Exit(1)
    }
    defer outFile.Close()

    const bufferSize = 64 * 1024
    inBuffer := make([]byte, bufferSize)
    outBuffer := make([]byte, bufferSize)

    strm := C.bz2alloc()
    defer C.bz2free(strm)

    if ret := C.BZ2_bzCompressInit(strm, 9, 0, 0); ret != C.BZ_OK {
        fmt.Fprintf(os.Stderr, "BZ2_bzCompressInit failed: %d\n", ret)
        os.Exit(1)
    }
    defer C.BZ2_bzCompressEnd(strm)

    for {
        inLen, err := inFile.Read(inBuffer)
        if err != nil && err != io.EOF {
            fmt.Fprintf(os.Stderr, "Error reading input file: %v\n", err)
            os.Exit(1)
        }

        if inLen == 0 {
            break
        }

        inLenC := C.uint(inLen)
        outLenC := C.uint(bufferSize)

        ret := C.bz2compress(strm, C.BZ_RUN,
            (*C.char)(unsafe.Pointer(&inBuffer[0])), &inLenC,
            (*C.char)(unsafe.Pointer(&outBuffer[0])), &outLenC)

        if ret != C.BZ_RUN_OK {
            fmt.Fprintf(os.Stderr, "BZ2_bzCompress failed: %d\n", ret)
            os.Exit(1)
        }

        if _, err := outFile.Write(outBuffer[:outLenC]); err != nil {
            fmt.Fprintf(os.Stderr, "Error writing to output file: %v\n", err)
            os.Exit(1)
        }
    }

    fmt.Println("Compression completed successfully.")
}

Building and Running the Application

Initialize the Go Module

   go mod init project-directory

Build the Project

   go build

Run the Application

   ./project-directory

Ensure input.txt exists before running the program. The compressed output will be saved as output.bz2.

Conclusion

By following this tutorial, you have successfully integrated C code with Go using cgo. Keeping the C code separate improves maintainability and allows better modularity. This approach can be extended to use other C libraries in Go applications efficiently.

If you have any questions or suggestions, feel free to leave a comment below!