C# Image Resizer Using ZeroMQ and Protobuf

Following on from the previous post I decided to attempt to use protobuf rather than json to transmit the data between the producer and the consumers. The updated repo can be found [C# Image Resizer Using ZeroMQ and Protobuf].(https://github.com/deltastateonline/ImageProcessor/tree/Using-Protobuf) I found this starting point on using protobuf in c# from Fast and efficient data serialisation with Protocol buffers (protobuf) in .NET. The first step is to add the protobuf-net package then define a class for the Image data definition. This class definition is an exact duplicate of the original, but with the required protobuf annotations. using ProtoBuf; namespace Common { [ProtoContract] public class ImageDefProto { [ProtoMember(1)] public required string Id { get; set; } [ProtoMember(2)] public required string Filename { get; set; } [ProtoMember(3)] public required string InputFolder { get; set; } [ProtoMember(4)] public required string OutputFolder { get; set; } [ProtoMember(5)] public decimal Resize { get; set; } } } The next step is to create a helper class which is used to serialize and deserialize the class into a byte array and from a byte array via a memory stream. namespace Common { public static class ProtobufHelpers { public static byte[] SerializeObject(T record) where T : class { using (var stream = new MemoryStream()) { Serializer.Serialize(stream, record); return stream.ToArray(); } } public static T DeserializeObject(byte[] data) { T obj = default(T); using (var stream = new MemoryStream(data)) { obj = Serializer.Deserialize(stream); } return obj; } } } In the ImageListing project, the image definition object has to be serialized as follows and published. var msg = new ImageDefProto { Id = messageId.ToString(), Filename = file, InputFolder = path, OutputFolder = outputPath, Resize = resize }; string encodedText = Convert.ToBase64String(ProtobufHelpers.SerializeObject(msg)); publisher.SendFrame(encodedText); While in the ImageProcessor project the data is deserialized and consumed. if (!subscriber.TryReceiveFrameString(TimeSpan.FromMilliseconds(5000), out string receivedFrame)) { Console.WriteLine("Timeout reached, no more messages. Exiting..."); break; } byte[] textBytes = Convert.FromBase64String(receivedFrame); var imageDetails = ProtobufHelpers.DeserializeObject(textBytes); Console.WriteLine(JsonSerializer.Serialize(imageDetails, jsonSerializerOptions)); await ResizeImage(imageDetails); Conclusion In this implmentation, it can be seen that using protobuf saves about 32% of the space required to transmit the data. This may not seem like much but in high volume systems and systems with limited resources, protobuf would be an excellent option. Let this post be a practical example for using protobuf. There are a number of reasons why you would choose using json over protobuf or vice versa - Ask ChatGPT In summary. Using Json length of string = 305; eyJJZCI6IjAiLCJGaWxlbmFtZSI6IkM6XFxVc2Vyc1xcb21vXFxQaWN0dXJlc1xcMjAyNS0wMy0wMS1mb290YmFsbFxcSU1HXzQxOTUuSlBHIiwiSW5wdXRGb2xkZXIiOiJDOlxcVXNlcnNcXG9tb1xcUGljdHVyZXNcXDIwMjUtMDMtMDEtZm9vdGJhbGwiLCJPdXRwdXRGb2xkZXIiOiJDOlxcVXNlcnNcXG9tb1xcUGljdHVyZXNcXDIwMjUtMDMtMDEtZm9vdGJhbGxfQ1NoYXJwIiwiUmVzaXplIjoyMH0= Using Protobuf length of string = 209; CgEwEjZDOlxVc2Vyc1xvbW9cUGljdHVyZXNcMjAyNS0wMy0wMS1mb290YmFsbFxJTUdfNDE5NS5KUEcaKUM6XFVzZXJzXG9tb1xQaWN0dXJlc1wyMDI1LTAzLTAxLWZvb3RiYWxsIjBDOlxVc2Vyc1xvbW9cUGljdHVyZXNcMjAyNS0wMy0wMS1mb290YmFsbF9DU2hhcnAqAggU Please like and leave a comment on this post. Next feature would be to implement commandline parameters and options for the ImageListing project.

May 9, 2025 - 08:19

C# Image Resizer Using ZeroMQ and Protobuf

Following on from the previous post I decided to attempt to use protobuf rather than json to transmit the data between the producer and the consumers. The updated repo can be found [C# Image Resizer Using ZeroMQ and Protobuf].(https://github.com/deltastateonline/ImageProcessor/tree/Using-Protobuf)

I found this starting point on using protobuf in c# from Fast and efficient data serialisation with Protocol buffers (protobuf) in .NET.

The first step is to add the protobuf-net package then define a class for the Image data definition. This class definition is an exact duplicate of the original, but with the required protobuf annotations.

using ProtoBuf;

namespace Common
{
    [ProtoContract]
    public class ImageDefProto
    {
        [ProtoMember(1)]
        public required string Id { get; set; }
        [ProtoMember(2)]
        public required string Filename { get; set; }
        [ProtoMember(3)]
        public required string InputFolder { get; set; }
        [ProtoMember(4)]
        public required string OutputFolder { get; set; }
        [ProtoMember(5)]
        public decimal Resize { get; set; }
    }
}

The next step is to create a helper class which is used to serialize and deserialize the class into a byte array and from a byte array via a memory stream.

namespace Common
{
   public  static class ProtobufHelpers
    {
        public static byte[] SerializeObject<T>(T record) where T : class
        {
            using (var stream = new MemoryStream())
            {
                Serializer.Serialize(stream, record);
                return stream.ToArray();
            }
        }
        public static T DeserializeObject<T>(byte[] data)
        {
            T obj = default(T);
            using (var stream = new MemoryStream(data))
            {
                obj = Serializer.Deserialize<T>(stream);
            }
            return obj;
        }
    }
}

In the ImageListing project, the image definition object has to be serialized as follows and published.

var msg = new ImageDefProto
{
    Id = messageId.ToString(),
    Filename = file,
    InputFolder = path,
    OutputFolder = outputPath,
    Resize = resize
};

string encodedText = Convert.ToBase64String(ProtobufHelpers.SerializeObject(msg));
publisher.SendFrame(encodedText);

While in the ImageProcessor project the data is deserialized and consumed.

if (!subscriber.TryReceiveFrameString(TimeSpan.FromMilliseconds(5000), out string receivedFrame))
{
    Console.WriteLine("Timeout reached, no more messages. Exiting...");
    break;
}

byte[] textBytes = Convert.FromBase64String(receivedFrame); 
var imageDetails = ProtobufHelpers.DeserializeObject<ImageDefProto>(textBytes);  

Console.WriteLine(JsonSerializer.Serialize(imageDetails, jsonSerializerOptions));
await ResizeImage(imageDetails);

Conclusion

In this implmentation, it can be seen that using protobuf saves about 32% of the space required to transmit the data. This may not seem like much but in high volume systems and systems with limited resources, protobuf would be an excellent option.

Let this post be a practical example for using protobuf.

There are a number of reasons why you would choose using json over protobuf or vice versa -

Ask ChatGPT

In summary.

Using Json

length of string = 305;

eyJJZCI6IjAiLCJGaWxlbmFtZSI6IkM6XFxVc2Vyc1xcb21vXFxQaWN0dXJlc1xcMjAyNS0wMy0wMS1mb290YmFsbFxcSU1HXzQxOTUuSlBHIiwiSW5wdXRGb2xkZXIiOiJDOlxcVXNlcnNcXG9tb1xcUGljdHVyZXNcXDIwMjUtMDMtMDEtZm9vdGJhbGwiLCJPdXRwdXRGb2xkZXIiOiJDOlxcVXNlcnNcXG9tb1xcUGljdHVyZXNcXDIwMjUtMDMtMDEtZm9vdGJhbGxfQ1NoYXJwIiwiUmVzaXplIjoyMH0=

Using Protobuf

length of string = 209;

CgEwEjZDOlxVc2Vyc1xvbW9cUGljdHVyZXNcMjAyNS0wMy0wMS1mb290YmFsbFxJTUdfNDE5NS5KUEcaKUM6XFVzZXJzXG9tb1xQaWN0dXJlc1wyMDI1LTAzLTAxLWZvb3RiYWxsIjBDOlxVc2Vyc1xvbW9cUGljdHVyZXNcMjAyNS0wMy0wMS1mb290YmFsbF9DU2hhcnAqAggU

Please like and leave a comment on this post.

Next feature would be to implement commandline parameters and options for the ImageListing project.