Gophers Swimming In the Sea (cgo)

tldr; Check out tblyler/go-mcrypt

Cgo enables the creation of Go packages that call C code.

Why?

Is there a really a good reason why someone would want to use C with Go? You lose easy cross-platform building (no more go get or go build without concern for extenal dependencies), dependency indepence, awkward memory copying, and the list goes on. Therefore, I advocate to only use cgo as a last resort to keep things the Go way, “simple, reliable, and efficient.” However, like most rules, there are exceptions.

Migrating Old Code

Go is a bit young to say that really has old code, but that doesn’t mean that you aren’t rewriting some backend code from a language like PHP to Go. In short, in my spare time, I have rewritten some proprietary network libraries from PHP to Go. Initially, the conversion was seamless. Need a network socket? No problem, there is the standard net package in Go for dead simple networking. Need to perform some certificate handshaking? crypto/rsa, crypto/x509, and encoding/pem are more than easy enough to deal with certificates. Nonetheless, there was an issue: the use of Rijndael 256 in PHP’s mcrypt functions with a 32 byte block size for symmetric encryption.

Rijndael in Go (or not)

A quick search of “Rijndael encryption algorithm” Will bring you to Wikipedia’s article on the Advanced Encryption Standard (AES). Which will quickly tell you…

AES operates on a 4×4 column-major order matrix of bytes, termed the state, although some versions of Rijndael have a larger block size and have additional columns in the state

I checkout out Go’s package list but only found crypto/aes, which, by definition, does not support a 32 byte block size like Rijdael 256. My first instinct was to take the time to rewrite Rijndael 256 from the ground up in Go while looking at C code for reference. However, after pulling up a simple example of the algorithm, I quickly realized this would take much longer than a weekend to get this code migration finished. I knew Go was able to call C code natively, so I decided to Go down the Gopher hole.

Cgo

A quick glance at Go’s cgo page draws your attention to a couple of simple examples that gives you the basic idea behind implenting C in Go…

// #cgo CFLAGS: -DPNG_DEBUG=1
// #cgo amd64 386 CFLAGS: -DX86=1
// #cgo LDFLAGS: -lpng
// #include <png.h>
import "C"

“Well that seems simple enough” and sure enough, it is that simple. So I made my own stub…

package main

/*
char* helloWorld() {
	return "Hello World!";
}
*/
import "C"

import "fmt"

func main() {
	// get the C string
	cString := C.helloWorld()

	// convert the C String to a Go string and print it
	fmt.Println(C.GoString(cString))
}

Which simply prints “Hello World!” to STDOUT. So I decided that it would just be easiest if I took the time to install libmcrypt on my machine, include it in C, and run all of the Rijndael 256 encryption in C.

Go-like C

I’m by no means, an expert when it comes to writing C, but I know the basics. Therefore, I tried to keep it as close to Go as possible when including libmcrypt. I will admit that I could have made the implementation more dynamic… allow different encryption aglorithims, encryption modes, or even implement it as a Cipher.Block. I wanted this done fast, and had no intentions of using libmcrypt for any other function, so I strictly implemented Rijindael 256 encrypt/decrypt wrapper functions in C.

mcrypt.go

#include <stdlib.h>
#include <string.h>
#include <mcrypt.h>
#define FAILED_MCRYPT_MODULE 1
#define INVALID_KEY_LENGTH 2
#define INVALID_IV_LENGTH 3
#define FAILED_TO_ENCRYPT_DATA 4
#define FAILED_TO_DECRYPT_DATA 5
#define INVALID_DATA_LENGTH 6
// getError convert a given error code to its string representation
const char* getError(int err) {
	switch (err) {
	case FAILED_MCRYPT_MODULE:
		return "Failed to open mcrypt module";
	case INVALID_KEY_LENGTH:
		return "Invalid key length";
	case INVALID_IV_LENGTH:
		return "Invalid iv length";
	case FAILED_TO_ENCRYPT_DATA:
		return "Failed to encrypt data";
	case FAILED_TO_DECRYPT_DATA:
		return "Failed to decrypt data";
	case INVALID_DATA_LENGTH:
		return "Invalid data length";
	}
	return mcrypt_strerror(err);
}

This is how I decided to store and interpret errors for the C code in Go. Pass an int reference to C wrapper calls, see if it was changed to a non zero value, and fetch the error message. This bypasses the need for awkward structs between Go and C.

mcrypt.go

...
// encrypt encrypt a given data set with rijndael 256
char* encrypt(void* key, int keyLength, void* iv, int ivLength, char* data, int* length, int* err) {
	if (*length <= 0) {
		*err = INVALID_DATA_LENGTH;
		return NULL;
	}
	...
	// allocate and copy the data to the output value
	char* output = malloc(sizeof *output * newLength);
	// append byte zeroes to the output array if needed
	for (i = *length; i < newLength; ++i) {
		output[i] = 0;
	}
	memcpy(output, data, *length);
	// update the length to the reallocated length
	*length = newLength;
	// loop through the output data by blockSize at a time
	for (i = 0; i < *length; i += blockSize) {
		// encrypt the block of output[i] plus blockSize
		if (mcrypt_generic(td, output+i, blockSize)) {
			*err = FAILED_TO_ENCRYPT_DATA;
			mcrypt_generic_deinit(td);
			mcrypt_module_close(td);
			free(output);
			return NULL;
		}
	}
	// finish up mcrypt
	mcrypt_generic_deinit(td);
	mcrypt_module_close(td);
	// return the encrypted data
	return output;
}

“Why are you passing so many parameters by reference?” Well, as it seemed at this writing, the easiest way to send data back to Go isn’t really by structs, but by altering the values passed by reference.

mcrypt.go

...
// decrypt decrypt a given data set with rijndael 256
char* decrypt(void* key, int keyLength, void* iv, int ivLength, char* data, int* length, int* err) {
	MCRYPT td = mcrypt_module_open("rijndael-256", NULL, "cbc", NULL);
	if (td == MCRYPT_FAILED) {
		*err = FAILED_MCRYPT_MODULE;
		mcrypt_module_close(td);
		return NULL;
	}
	...
	// allocate and copy the data to the output value
	char* output = malloc(sizeof *output * *length);
	memcpy(output, data, *length);
	// loop through the output data by blockSize at a time
	for (i = 0; i < *length; i += blockSize) {
		// decrypt the block of output[i] plus blockSize
		if (mdecrypt_generic(td, output+i, blockSize)) {
			*err = FAILED_TO_DECRYPT_DATA;
			mcrypt_generic_deinit(td);
			mcrypt_module_close(td);
			free(output);
			return NULL;
		}
	}
	// finish up mcrypt
	mcrypt_generic_deinit(td);
	mcrypt_module_close(td);
	// return the decrypted data
	return output;

And the decryption wrapper is nearly identical.

Letting the Go(phers) Out In the Sea

Per the previous stub, it’s as easy as wrapping taking all of the C code into a block comment followed immediately by import "C". Thanks to the simple nature of the wrappers, the Go functions do not require much logic.

mcrypt.go

package mcrypt

/*
#cgo LDFLAGS: -lmcrypt
...
import "C"

import (
	"errors"
	"unsafe"
)
...
// Encrypt encrypt something with mcrypt rijndael-256 PHP-style
func Encrypt(key []byte, iv []byte, data []byte) ([]byte, error) {
	// keep track of the size of the input data
	length := C.int(len(data))
	if length == 0 {
		return nil, errors.New("Invalid data size of 0")
	}
	// keep track of any errors that occur on encryption
	err := C.int(0)
	// encrypt the data
	encryptedData := C.encrypt(unsafe.Pointer(&key[0]), C.int(len(key)), unsafe.Pointer(&iv[0]), C.int(len(iv)), (*C.char)(unsafe.Pointer(&data[0])), (*C.int)(unsafe.Pointer(&length)), (*C.int)(unsafe.Pointer(&err)))

	// if err is not 0, there is an error
	if int(err) != 0 {
		return nil, errors.New(C.GoString(C.getError(err)))
	}

	// ensure that memory is freed on the encrypted data after it is converted to Go bytes
	defer C.free(unsafe.Pointer(encryptedData))

	// return the Go bytes of the encrypted data
	return C.GoBytes(unsafe.Pointer(encryptedData), length), nil
}

What I did here was convert certain parameters to types that can be interpretted by C. I also take advantage of Go’s unsafe package, which allows you to pass pointers to C code seamlessly (therefore, allowing C to manipulate the data it points to). As I passed the unsafe.Pointer’s, I casted them to their appropriate C types and if it was a slice, I made sure to have it point to the first element. Interestingly enough, there is some memory management you need to perform with this code, and it can be seen by the following line…

defer C.free(unsafe.Pointer(encryptedData))

This ensures that the char array from C returned by the wrapper functions is released from memory after Go convertes it to a byte slice.

Similary to the C wrapper functions, the Go decrypt function is nearly identical to its encrypt function…

mcrypt.go

// Decrypt decrypt something with mcrypt rijndael-256 PHP-style
func Decrypt(key []byte, iv []byte, data []byte) ([]byte, error) {
	// keep track of the size of the input data
	length := C.int(len(data))
	if length == 0 {
		return nil, errors.New("Invalid data size of 0")
	}
	// keep track of any errors that occur on decryption
	err := C.int(0)
	// decrypt the data
	decryptedData := C.decrypt(unsafe.Pointer(&key[0]), C.int(len(key)), unsafe.Pointer(&iv[0]), C.int(len(iv)), (*C.char)(unsafe.Pointer(&data[0])), (*C.int)(unsafe.Pointer(&length)), (*C.int)(unsafe.Pointer(&err)))

	// if err is not 0, there is an error
	if int(err) != 0 {
		return nil, errors.New(C.GoString(C.getError(err)))
	}

	// ensure that memory is freed on the decrypted data after it is converted to Go bytes
	defer C.free(unsafe.Pointer(decryptedData))

	// return the Go bytes of the decrypted data
	return C.GoBytes(unsafe.Pointer(decryptedData), length), nil
}

Looking Forward

You are able to find the doc page at this link. There is a basic unit test for this package (mcrypt_test.go) that I threw together. As you can see, it ends up being a simple Encrypt/Decrypt function that simply ask for a key, iv, and data byte slices returning the encrypted/decrypted data with an error. Ideally, cgo would be eliminated from my solution and a rijndeal 256 implementation would be made for Go. Thankfully, I was able to gain a noticeable speed increase with my cgo implementation compared to PHP’s mcrypt implementaiton regardless. My project was strictly for one platform so the loss of easy cross-platform compliation was not of great concern. All in all, cgo is a great tool to have in your Go toolbelt.