How to: use a shared protobuf schema (in golang)

After being tired of encoding and decoding JSON, some developers turned to protocol buffer to serialize data in communications between servers to gain latency improvement due to its payload after serialization is much more byte-saving than JSON.

However, protobuf is not self-explanatory, which means you need a .proto file defining a server’s schema before you can interact with it, or otherwise, you cannot interpret their messages. Service providers share their schemes, and you need to integrate them into your code. This lead to our question, how to use such a .proto file.

We will do it for a golang project.

First, you need to have the protobuf compiler installed on your computer. If you haven’t, download one of the compiled packages that match your platform from Github releases, and extract the binary protoc executable into /usr/local/bin or any other path included in $PATH. And don’t forget to install its golang plugin as well.

1
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest

Next, we will generate code from a protobuf schema, i.e. a .proto file. You can find such a file from your service provider if they support protobuf.

1
protoc --proto_path=src_dir --go_out=out_dir foo.proto

This will generate a set of codes in out_dir.

If the schema didn’t define go_package by itself, then you should pass the argument through the command line.

1
protoc --proto_path=src_dir --go_out=out_dir --go_opt=Mfoo.proto=./foo foo.proto

The --go_opt=Mfoo.proto= specifies what import path your generated code will have relative to the out_dir. If you didn’t specify the parameter, you would get this error:

protoc-gen-go: unable to determine Go import path for “foo.proto”

Please specify either:
• a “go_package” option in the .proto source file, or
• a “M” argument on the command line.

See https://developers.google.com/protocol-buffers/docs/reference/go-generated#package for more information.

Now, you have generated golang code to serialize and deserialize data according to the predefined schema. All you have to do with it is to import the code into your own packages.

1
2
3
import (
"github.com/you/project/out_dir/foo"
)