JSON Voorhees
Killer JSON for C++
Loading...
Searching...
No Matches
Serialization Builder DSL

Most applications tend to have a lot of structure types.

While it is possible to write a deserializer and serializer (or adapter) for each type, this can get a little bit tedious. Beyond that, it is very difficult to look at the contents of adapter code and discover what the JSON might actually look like. The builder DSL is meant to solve these issues by providing a convenient way to describe conversion operations for your C++ types.

At the end of the day, the goal is to take some C++ structures like this:

struct person
{
std::string first_name;
std::string last_name;
int age;
std::string role;
};
struct company
{
std::string name;
bool certified;
std::vector<person> employees;
std::list<person> candidates;
};

...and easily convert it to an from a JSON representation that looks like this:

{
"name": "Paul's Construction",
"certified": false,
"employees": [
{
"first_name": "Bob",
"last_name": "Builder",
"age": 29
},
{
"first_name": "James",
"last_name": "Johnson",
"age": 38,
"role": "Foreman"
}
],
"candidates": [
{
"firstname": "Adam",
"lastname": "Ant"
}
]
}

To define a formats for this person type using the serialization builder DSL, you would say:

.type<person>()
.member("first_name", &person::first_name)
.alias("firstname")
.member("last_name", &person::last_name)
.alias("lastname")
.member("age", &person::age)
.until({ 6,1 })
.default_value(21)
.default_on_null()
.check([] (int value) { if (value < 0) throw std::logic_error("Age must be positive."); })
.member("role", &person::role)
.since({ 2,0 })
.default_value("Builder")
.type<company>()
.member("name", &company::name)
.member("certified", &company::certified)
.member("employees", &company::employees)
.member("candidates", &company::candidates)
.register_containers<person, std::vector, std::list>()
.check_references(jsonv::formats::defaults())
;
Simply put, this class is a collection of deserializer and serializer instances.
Definition formats.hpp:157
static formats defaults()
Get the default formats instance.

Deserialization

A type described with the DSL is deserialized by walking its JSON object's keys in the order the document wrote them, handing each one to the member which claims it – by the name the member was declared with, or by one of its aliases. Members are deserialized in document order rather than declaration order, so that is also the order a member's check and setter run in and the order problems are reported in. Once the walk reaches the end of the object, each member no key claimed is given its default value or, if it has none, reported as missing; that pass, being over the members rather than the keys, goes in declaration order. A member whose key held a null under default_on_null is given its default in the same pass, since that is what the null stands for.

A key which no member claims is skipped rather than collected: its value is stepped over unread, in a single step however large it is, and never built into a value nobody asked for. Its name is all that is kept, and only when something needs it. An on_unknown_members handler is handed the names of all of them once the walk is done. And a deserialize_options::on_duplicate_key of duplicate_key_action::exception remembers every key, claimed or not, so that the object is refused if one comes round again.

No hook is handed the JSON being read as an argument, but every one which takes a deserialization_context can ask it for the object. deserialization_context::source_value has it as a value to read members out of: for pre_deserialize, before the walk, and for on_unknown_members, every default_value and post_deserialize after it. Once the walk has reached the object's }, deserialization_context::encoded_source quotes the whole of it too, which is what a hook validating the object wants to put in its message. Neither is worked out until a hook asks, so they cost a deserialization which never asks nothing.

Serialization

A type described with the DSL is serialized as an object, written into the writer one member at a time in the order the members were declared: each member's key, then its value through the serializer registered for the member's type. A member whose serialize_if – or the since, until, after and before built on it – says to leave it out is skipped, key and all. The DSL builds nothing in between, so what a member's serializer writes reaches the sink as it is written. A value made with to_json holds the same members sorted by key, as every value does, so the text written directly and the text of that value differ in the order of their members and in nothing else. For the person above, once the adapters are combined with the defaults that check_references checked them against:

person bob{ "Bob", "Builder", 29, "Foreman" };
std::string text = jsonv::serialize(bob, with_defaults);
// {"first_name":"Bob","last_name":"Builder","age":29,"role":"Foreman"}
std::string sorted = jsonv::to_string(jsonv::to_json(bob, with_defaults));
// {"age":29,"first_name":"Bob","last_name":"Builder","role":"Foreman"}
static formats compose(const list &bases)
Create a new (empty) formats using the bases as backing formats.
std::string serialize(const T &from, const formats &fmts=formats::global())
Serialize from into JSON text using fmts (by default jsonv::formats::global()).
value to_json(const T &from, const formats &fmts)
Encode a JSON value from from using the provided fmts.
JSONV_PUBLIC std::string to_string(const ast_node_type &type)
Get what operator<< writes for type as a std::string.

A member's since and until ask the serialization_context which version is being written, and with no version every member is written. Serialize through a context made with one to leave out the members it rules out:

jsonv::serialization_context version_1(with_defaults, jsonv::version(1, 0));
std::string before_role = jsonv::serialize(bob, version_1);
// {"first_name":"Bob","last_name":"Builder","age":29}
Provides extra information to routines used for serialization: the formats to find other serializers ...
Definition serialize.hpp:42
Represents a version used to deserialize and encode JSON objects from C++ classes.
Definition version.hpp:24

A failure inside a member is reported where it happened: the serialization_error carries the member's path, and a member of a member described with the DSL extends it – .address.city for the city of an address.

Reference

The DSL is made up of three major parts:

  1. formats – modifies a jsonv::formats object by adding new type adapters to it
  2. type – modifies the behavior of a jsonv::adapter by adding new members to it
  3. member – modifies an individual member inside of a specific type

Each successive function call transforms your context. Narrowing calls make your context more specific; for example, calling type from a formats context allows you to modify a specific type. Widening calls make the context less specific and are always available; for example, when in the member context, you can still call type from the formats context to specify a new type.

Formats Context

Commands in this section modify the behavior of the underlying jsonv::formats object.

Level

check_references

  • check_references(formats)
  • check_references(formats, std::string name)
  • check_references(formats::list)
  • check_references(formats::list, std::string name)
  • check_references()
  • check_references(std::string name)

Tests that every type referenced by the members of the output of the DSL have a deserializer and a serializer. The provided formats is used to draw extra types from (a common value is jsonv::formats::defaults). In other words, it asks the question: If the formats from this DSL was combined with these other formats, could all of the types be encoded and decoded?

This does not mutate the DSL in any way. On successful verification, it will appear that nothing happened. If the verification is not successful, an exception will be thrown with the offending types in the message. For example:

There are 2 types referenced that the formats do not know how to serialize:
- date_type (referenced by: name_space::foo, other::name::space::bar)
- tree

If name is provided, the value will be output to the error message on failure. This can be useful if you have multiple check_references statements and wish to more easily determine the failing formats combination from the error message alone.

Note
This is evaluated immediately, so it is best to call this function as the very last step in the DSL.
.check_references(jsonv::formats::defaults())

reference_type

  • reference_type(std::type_index type)
  • reference_type(std::type_index type, std::type_index from)

Explicitly add a reference to the provided type in the DSL. If from is provided, also add a back reference for tracking purposes. The from field is useful for tracking why the type is referenced.

Type references are used in check_references to both check and generate error messages if the formats the DSL is building cannot fully create and deserialize JSON values. You do not usually have to call this, as each call to member calls this automatically.

.reference_type(std::type_index(typeid(int)), std::type_index(typeid(my_type)))
.reference_type(std::type_index(typeid(my_type))

register_adapter

  • register_adapter(const adapter*)
  • register_adapter(std::shared_ptr<const adapter>)

Register an arbitrary adapter with the formats we are currently building. This is useful for integrating with type adapters that do not (or can not) use the DSL.

.register_adapter(my_type::get_adapter())

register_optional

  • register_optional<TOptional>()

Similar to register_adapter, but automatically create an optional_adapter<TOptional> to store.

.register_optional<std::optional<int>>()
.register_optional<boost::optional<double>>()

register_container

  • register_container<TContainer>()

Similar to register_adapter, but automatically create a container_adapter<TContainer> to store.

.register_container<std::vector<int>>()
.register_container<std::list<std::string>>()

register_containers

  • register_containers<T, template <class...> class... TTContainer>

Convenience function for calling register_container for multiple containers with the same value_type. Unfortunately, it only supports varying the first template parameter of the TTContainer types, so if you wish to do something like vary the allocator, you will have to either call register_container multiple times or use a template alias.

.register_containers<int, std::list, std::deque>()
.register_containers<double, std::vector, std::set>()

register_wrapper

  • register_wrapper<TWrapper>()

Similar to register_adapter, but automatically create an wrapper_adapter<TWrapper> to store.

.register_optional<std::optional<int>>()
.register_optional<boost::optional<double>>()

enum_type

  • enum_type<TEnum>(std::string name, std::initializer_list<std::pair<TEnum, jsonv::value>>)
  • enum_type_icase<TEnum>(std::string name, std::initializer_list<std::pair<TEnum, jsonv::value>>)

Create an adapter for the TEnum type with a mapping of C++ values to JSON values and vice versa. The most common use of this is to map enum values in C++ to string representations in JSON. TEnum is not restricted to types which are enum, but can be anything which you would like to restrict to a limited subset of possible values. Likewise, JSON representations are not restricted to being of kind::string.

The sibling function enum_type_icase will create an adapter which uses case-insensitive checking when converting to C++ values in deserialize.

.enum_type<ring>("ring",
{
{ ring::fire, "fire" },
{ ring::wind, "wind" },
{ ring::earth, "earth" },
{ ring::water, "water" },
{ ring::heart, "heart" }, // "heart" is preferred for to_json
{ ring::heart, "useless" }, // "useless" is read as ring::heart
{ ring::fire, 1 }, // the JSON value 1 is also read as ring::fire
{ ring::ussr, "wind" }, // old C++ value ring::ussr will get output as "wind"
}
)
.enum_type_icase<int>("integer",
{
{ 0, "zero" },
{ 0, "naught" },
{ 1, "one" },
{ 2, "two" },
{ 3, "three" },
}
)
See also
enum_adapter

polymorphic_type

  • polymorphic_type<<TPointer>(std::string discrimination_key);

Create an adapter for the TPointer type (usually std::shared_ptr or std::unique_ptr) that knows how to serialize and deserialize one or more types that can be polymorphically represented by TPointer, i.e. derived types. It uses a discrimination key to determine which concrete type should be instantiated when deserializing values from json.

.polymorphic_type<std::unique_ptr<base>>("type")
.subtype<derived_1>("derived_1")
.subtype<derived_2>("derived_2", keyed_subtype_action::check)
.subtype<derived_3>("derived_3", keyed_subtype_action::insert);
@ check
Ensure the correct key/value pair was inserted by serialization. Throws std::runtime_error if it wasn...
@ insert
Insert the correct key/value pair as part of serialization.

The keyed_subtype_action can be used to configure the adapter to make sure that the discrimination key was correctly serialized (keyed_subtype_action::check) or to insert the discrimination key for the underlying type so that the underlying type doesn't need to do that itself (keyed_subtype_action::insert). The default is to do nothing (keyed_subtype_action::none). Either action needs the finished object, so it builds the subtype as a value before writing it, and that subtype's members are written sorted by key rather than in the order they were declared.

extend

Extend the formats_builder with the provided func by passing the current builder to it. This provides a more convenient way to call helper functions.

foo(builder);
bar(builder);
baz(builder);

This can be done equivalently with:

.extend(foo)
.extend(bar)
.extend(baz)

on_duplicate_type

  • on_duplicate_type(on_duplicate_type_action action);

Set what action to take when attempting to register an adapter, but there is already an adapter for that type in the formats. The default is to throw a duplicate_type_error exception (duplicate_type_action::exception), but the formats_builder can also be configured to ignore the duplicate (duplicate_type_action::ignore), or to replace the existing adapter with the new one (duplicate_type_action::replace). This is useful when calling multiple extend methods that may add common types to the formats_builder.

Narrowing

type<T>

Create an adapter for type T and begin building the members for it. If func is provided, it will be called with the adapter_builder<T> this call to type creates, which can be used for creating common extension functions.

.type<my_type>()
.member(...)
.
.
.

Type Context

Commands in this section modify the behavior of the jsonv::adapter for a particular type.

Level

pre_deserialize

Call the given perform function during the deserialize operation, but before performing any deserialization. This can be called multiple times – all functions will be called in the order they are provided.

The source document is not among the arguments, but deserialization_context::source_value has it, before any member has been read out of it. Nothing yet knows that it is an object, so it is shown as whatever it is – including a null that type_default_on_null goes on to replace. Its text cannot be quoted yet, so deserialization_context::encoded_source is empty.

.type<my_type>()
.pre_deserialize([] (deserialization_context& context)
{
if (context.source_value().value().at("schema") != jsonv::value(2))
throw std::invalid_argument("Only schema 2 is supported");
}
)
Represents a single JSON value, which can be any one of a potential kind, each behaving slightly diff...
Definition value.hpp:113

post_deserialize

Call the given perform function after the deserialize operation. All functions will be called in the order they are provided. This allows validation methods to be called on the deserialized object as part of deserialization. Postprocessing functions are allowed to mutate the deserialized object.

The JSON the object was read from is there to read, through deserialization_context::source_value, and a validation which fails can quote it through deserialization_context::encoded_source:

.type<my_type>()
.member("low", &my_type::low)
.member("high", &my_type::high)
.post_deserialize([] (deserialization_context& context, my_type&& out)
{
if (out.high < out.low)
throw std::invalid_argument("high is below low in "
+ std::string(context.encoded_source())
);
return std::move(out);
}
)

type_default_on_null

  • type_default_on_null()
  • type_default_on_null(bool on)

If the JSON value null is in the input, should this type take on some default? This option is only considered if a type_default_value was provided.

type_default_value

What value should be used to create the default for this type? It stands in for the null rather than reading it, so deserialization_context::source_value shows it nothing.

.type<my_type>()
.type_default_on_null()
.type_default_value(my_type("default"))

on_unknown_members

  • on_unknown_members(std::function<void (deserialization_context& context, std::set<std::string> unknown_members)> action )

When deserializing, perform some action if the object has members which no declared member claims. By default they are simply ignored, so this is useful if you wish to throw an exception (or anything you want). The action is handed the names of the keys which claimed no member. The walk stepped over their values rather than reading them, but the action can read them out of deserialization_context::source_value, or quote the object they are in with deserialization_context::encoded_source.

.type<my_type>()
.member("x", &my_type::x)
.member("y", &my_type::y)
.on_unknown_members([] (deserialization_context&, std::set<std::string> unknown_members)
{
throw unknown_members_error("my_type", std::move(unknown_members));
}
)

There is a convenience function named deny_unknown_members which does this for you.

.type<my_type>()
.member("x", &my_type::x)
.member("y", &my_type::y)
.on_unknown_members(jsonv::deny_unknown_members)

Narrowing

member

  • member(std::string name, TMember T::*selector)
  • member(std::string name, const TMember& (*access)(const T&), void (*mutate)(T&, TMember&&))
  • member(std::string name, const TMember& (T::*access)() const, TMember& (T::*mutable_access)())
  • member(std::string name, const TMember& (T::*access)() const, void (T::*mutate)(TMember))
  • member(std::string name, const TMember& (T::*access)() const, void (T::*mutate)(TMember&&))

Adds a member to the type we are currently building. By default, the member will be serialized with the key of the given name and the deserializer will search for the given name. If you wish to change properties of this field, use the Member Context. Members are written in the order they are declared, and declaring a second member with a name already declared for the type throws std::invalid_argument.

.type<my_type>()
.member("x", &my_type::x)
.member("y", &my_type::y)
.member("thing", &my_type::get_thing, &my_type::set_thing)

Member Context

Commands in this section modify the behavior of a particular member. Here, T refers to the containing type (the one we are adding a member to) and TMember refers to the type of the member we are modifying.

Level

after

  • after(version)

Only serialize this member if the serialization_context was not created with a version, or its version is greater than the provided version.

alias

  • alias(std::string name)

Provide another name to look for when deserializing this member. If a document provides values under more than one of a member's names, the earliest declared is preferred, starting with the name the member was declared with.

before

  • before(version)

Only serialize this member if the serialization_context was not created with a version, or its version is less than the provided version.

check

  • check(std::function<void (const TMember&)> inspect)
  • check(std::function<bool (const TMember&)> accept, std::function<void (const TMember&)> thrower )
  • check(std::function<bool (const TMember&)> accept, TException ex)

Checks the value read for this member before it reaches it. In the first form, inspect is expected to throw for itself. In the latter forms, a value accept returns false for is handed to thrower, or ex is thrown directly; ex may be anything which cannot be called with the value, which is what tells it from a thrower.

.member("x", &my_type::x)
.check([] (int x) { if (x < 0) throw std::logic_error("x must not be negative"); })
.check([] (int x) { return x < 100; },
[] (int x) { throw std::out_of_range("x must be below 100, not " + std::to_string(x)); }
)
.check([] (int x) { return x % 2 == 0; }, std::logic_error("x must be divisible by 2"))

default_value

Provide a default value for this member if no key is found when deserializing. The function implementation can synthesize the value however it likes. A missing key is only known to be missing once every key which was there has gone by, so defaults are taken once the walk is done – including one standing in for a null under default_on_null – and the function can read the rest of the object through deserialization_context::source_value, or quote it through deserialization_context::encoded_source. What it reads is the JSON. A default which depends on the members as they were deserialized belongs in post_deserialize, which sees the whole object once it is built.

.member("x", &my_type::x)
.default_value(10)
.member("name", &my_type::name)
.member("display_name", &my_type::display_name)
.default_value([] (deserialization_context& context)
{
return context.source_value().value().at("name").as_string();
}
)

default_on_null

  • default_on_null()
  • default_on_null(bool on)

If the value associated with this key is kind::null, should that be treated as though the key were missing, and the default value taken? This option is only considered if a default_value was provided.

serialize_if

Only serialize this member if the check function returns true.

since

  • since(version)

Only serialize this member if the serialization_context was not created with a version, or its version is greater than or equal to the provided version.

until

  • until(version)

Only serialize this member if the serialization_context was not created with a version, or its version is less than or equal to the provided version.