Published on

Custom Converters for JSON serialization in .NET

Authors
json converter
Photo by Laura Ockel on Unsplash

A month back, I found that some of the APIs of a payment aggregator needed custom behaviour which was a bit different from built-in converters. These led me to explore more on custom converters for JSON serialization in .NET.

  • Serialization (or Marshalling / Encoding) is the process of converting object to its string format (JSON / XML / etc.).
  • Deserialization (or Unmarshalling / Decoding) is the process of converting string (JSON / XML / etc.) to its object form.

1st Usecase: Deserialization

One of the payment aggregators at work were responding in a non-standard way in a HTTP API. This was causing a JSON parsing issue.

Unhandled exception. System.Text.Json.JsonException: The JSON value could not be converted to Response. Path: $[0].output.response | LineNumber: 5 | BytePositionInLine: 51.

The ideal response element would have been as follows

{
  "id": 2,
  "output": {
    "status": "ENABLED",
    "response": {
      "checkout": "express"
    }
  }
}

but the in some failure cases the response contained following element.

{
  "id": 2,
  "output": {
    "status": "DISABLED",
    "response": "{\"error\": \"DISABLED due to xyz.\"}"
  }
}

Even if the HTTP status code was 200 OK the response was as follows.

[
  {
    "id": 1,
    "output": {
      "status": "DISABLED",
      "response": "{error: DISABLED due to xyz.}"
    }
  },
  {
    "id": 2,
    "output": {
      "status": "ENABLED",
      "response": {
        "checkout": "express"
      }
    }
  }
]

This was handled using custom converter.

using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Text.Json.Serialization;

string apiResponse = $@"[
{{
   ""id"": ""1"",
    ""output"": {{
        ""status"": ""DISABLED"",
        ""response"": ""error: disabled due to xyz.""
    }}
}},
{{
   ""id"": ""2"",
    ""output"": {{
        ""status"": ""ENABLED"",
        ""response"":
       {{
           ""checkout"":""express""
       }}
    }}
}}
]";

var responseInstance = JsonSerializer.Deserialize<List<PaymentInstrument>>(
  apiResponse, new JsonSerializerOptions());
var checkout = responseInstance[1].Output.Response.Checkout;

Console.WriteLine(checkout);

var ar = JsonSerializer.Serialize<List<PaymentInstrument>>(responseInstance,
 new JsonSerializerOptions());
Console.WriteLine(ar);


public class PaymentInstrument
{
    [JsonPropertyName("id")]
    public string Id { get; set; }
    [JsonPropertyName("output")]
    public Output Output { get; set; }
}

public class Output
{
    [JsonPropertyName("status")]
    public string Status { get; set; }

    [JsonPropertyName("response")]
    [JsonConverter(typeof(ResponseConverter))]
    public Response Response { get; set; }
}

public class Response
{
    [JsonPropertyName("checkout")]
    public string Checkout { get; set; }
}


public class ResponseConverter : JsonConverter<Response>
{
    public override Response Read(ref Utf8JsonReader reader,
     Type typeToConvert, JsonSerializerOptions options)
    {
        var cr = new Response();
        if (reader.TokenType != JsonTokenType.StartObject)
        {
            return cr;
        }

        cr = JsonSerializer.Deserialize<Response>(ref reader);
        return cr;
    }

    public override void Write(Utf8JsonWriter writer,
    Response value, JsonSerializerOptions options)
    {
        JsonSerializer.Serialize(writer, value, options);
    }
}

express
[{"id":"1","output":{"status":"DISABLED","response":{"checkout":null}}},{"id":"2","output":{"status":"ENABLED","response":{"checkout":"express"}}}]

In ResponseConverter, Read method is overriden to ignore non-object (in this case string) value for deserialization. No changes were required in Write() so it uses the default Serialize() method.

2nd Usecase: Serialization

In another API for payment initiation, we were expected to send an object representation and not the serialized json format.

The correct request curl was

curl -X POST https://api.aggregator.com/trans
-d "order_id=:order_id" \
-d "metadata={"card_token":"123abc","xyz_offer_data":[{"method":"card","card_funding":"CREDIT","offer_id":"xyz"}]}

The data for this API was stored in a Dictionary<string, string>() so the request being formed was as follows.

curl -X POST https://api.aggregator.com/trans
-d "order_id=:order_id" \
-d "metadata={"card_token":"123abc","xyz_offer_data":"[{ \u0022method\u0022 : \u0022card\u0022, \u0022card_funding\u0022: \u0022CREDIT\u0022, \u0022offer_id\u0022: \u0022xyz\u0022}]}

\u0022 is the Unicode for double quote ".

This too was handled using custom converter.

using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Web;

const string OFFER_DATA = "xyz_offer_data";
var metadata = new Dictionary<string, string>{["card_token"] = "123abc"};
metadata[OFFER_DATA] = @"[{ ""method"" : ""card"", ""card_funding"": ""CREDIT"", ""offer_id"": ""xyz""}]";

Console.WriteLine("With default serialization");
var meta = JsonSerializer.Serialize(metadata);
Console.WriteLine(meta);
var options = new JsonSerializerOptions
{
    DefaultIgnoreCondition = JsonIgnoreCondition.Never,
    WriteIndented = false,
    Converters =
    {
        new CustomDictionaryJsonConverter()
    }
};

Console.WriteLine("With custom serialization");
meta = JsonSerializer.Serialize(metadata, options);
Console.WriteLine(meta);

var md = JsonSerializer.Deserialize<Dictionary<string, string>>(meta, options);
Console.WriteLine(md.GetValueOrDefault(OFFER_DATA, "-"));
Console.WriteLine(md.GetValueOrDefault("card_token", "-"));

public class CustomDictionaryJsonConverter : JsonConverter<Dictionary<string, string>>
{

    public override Dictionary<string, string> Read(
        ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        if (reader.TokenType != JsonTokenType.StartObject)
        {
            throw new JsonException($"JsonTokenType was of type {reader.TokenType}, only objects are supported");
        }

        var dictionary = new Dictionary<string, string>();
        while (reader.Read())
        {
            if (reader.TokenType == JsonTokenType.EndObject)
            {
                return dictionary;
            }

            if (reader.TokenType != JsonTokenType.PropertyName)
            {
                throw new JsonException("JsonTokenType was not PropertyName");
            }

            var propertyName = reader.GetString();

            if (string.IsNullOrWhiteSpace(propertyName))
            {
                throw new JsonException("Failed to get property name");
            }

            reader.Read();
            if (reader.TokenType == JsonTokenType.String)
            {
                dictionary.Add(propertyName, reader.GetString());
            }
            else if(reader.TokenType == JsonTokenType.StartArray)
            {
                // ignores array values while serialization
                while (reader.Read())
                {
                    if (reader.TokenType == JsonTokenType.EndArray)
                    {
                        break;
                    }
                }
            }

        }

        return dictionary;
    }

    public override void Write(
        Utf8JsonWriter writer, Dictionary<string, string?> value, JsonSerializerOptions options)
    {
        writer.WriteStartObject();
        foreach (KeyValuePair<string, string?> kv in value)
        {
            string propertyName = kv.Key;
            writer.WritePropertyName(propertyName);
            if (propertyName != "xyz_offer_data" || kv.Value == null)
            {
                writer.WriteStringValue(kv.Value);
                continue;
            }

            var cardOffer = JsonSerializer.Deserialize<List<Dictionary<string, string>>>(kv.Value);
            if (cardOffer == null)
            {
                writer.WriteStringValue(kv.Value);
                continue;
            }

            writer.WriteStartArray();
            foreach (var offer in cardOffer)
            {
                writer.WriteStartObject();
                foreach (var data in offer)
                {
                    writer.WritePropertyName(data.Key);
                    writer.WriteStringValue(data.Value);
                }
                writer.WriteEndObject();
            }
            writer.WriteEndArray();
        }

        writer.WriteEndObject();
    }
}

Output

With default serialization
{"card_token":"123abc","xyz_offer_data":"[{ \u0022method\u0022 : \u0022card\u0022, \u0022card_funding\u0022: \u0022CREDIT\u0022, \u0022offer_id\u0022: \u0022xyz\u0022}]"}
With custom serialization
{"card_token":"123abc","xyz_offer_data":[{"method":"card","card_funding":"CREDIT","offer_id":"xyz"}]}
-
123abc

Lets deep dive what's happening in Write() of custom converter.

  • WriteStartObject() start the object creation i.e. add the { to the deserialization instance.
  • You loop over the KeyValuePair
    • WritePropertyName() adds the property name i.e. "card_token": for the first pair.
    • Write the value of the pair if its not the key which contains xyz_offer_data property which needs custom handling.
    • You deserialize the offer data array contents ([{"method":"card","card_funding":"CREDIT","offer_id":"xyz"}]) to cardOffervariable.
    • Custom handling of cardOffer
      • WriteStartArray() add array i.e. adds [
      • Loop over the cardOfferarray,
        • Write each object e.g. "method":"card" for the first element pair.
      • EndStartArray() ends array i.e. adds ]
  • WriteEndObject() ends the object. i.e. adds }

In Read() method, only a string values are entered in the output Dictionary<string, string>(). If an array value is encountered i.e. reader.TokenType == JsonTokenType.StartArray, its ignored by looping over without adding in the output.

Summary

JsonConverter provides a low-level approach to customise json tokens. By analyzing property name and token types, you can customize the behaviour of your JSON serialization Read() and deserialization Write().

Resources