In this post we have tried wrapping up all of the currently discussed topics into two very light weight frameworks, that can easily be included into your own project. This provides hazzle-free Codable support, type safe paths and RxSwift extensions — and you can pick and choose the features you wish to use.

But before we start talking about the open sourced frameworks, let’s look at a topic that can make our lives a bit easier:

Automatic Code Generation

In the article about type-safe paths, we described a strategy for modelling a hierarchy using constrained extensions of the Path type. These extensions require typing quite a bit of boilerplate.

Tools like Sourcery and SwiftGen have become popular for a some very good reasons:

Boilerplate is boring and error prone to maintain.

Code that is automatically generated from some form of assets will stay up-to-date, so if you remove or rename an asset, you will immediately get a compile error if your code still refers to these assets.

So instead of having to type and maintain a huge list of phantom types and constrained Path extensions, why not just define your schema and let the code be auto-generated?

This will define Paths from the root of the structure and down to the leaf nodes which must correspond to names of model types in your code.

A json key that is wrapped in angle brackets means that the data at this point in the tree is part of a collection.

As you will notice, there is no entity defining a chatroom. For the sake of the above schema there is no model type corresponding to a chatroom, but rather you need to create a path to a chatroom in order to get to the messages of the chatroom.

This concept is modelled using phantom types. The code generator will generate an enum named Chatroom with no values. This means that the Chatroom can never be instantiated, but it can still be used as a generic restriction in our code.

With the code generated from the schema above, you can generate paths like:

let messagesPath = Path().chatrooms.child("firechat").messages

Or you can use the auto generated convenience method that wraps the .child(_ key: String) method as follows:

let messagesPath = Path().chatroom("firechat").messages

The type of the messagesPath variable is Path<Message>.Collection. In other words, a path to a collection of messages.

The full code generated from the above json is provided here as an example:

The Swift script that generates the Path extensions is called GeneratePaths.swift . I don’t think that it is very interesting to explore exactly what it does — but overall the strategy is:

The JSON structure is modelled using a single recursive type: Tree that can either represent a ‘leaf’ node (the concrete model types like Configuration, Message and String in the example) or ‘branch’ node (represented by the generated phantom types in the output — like Chatroom in the example).

In the decoding, some restructuring occurs after child nodes have been processed, since only after parsing the parent nodes can we tell whether or not the child nodes are part of a collection or not.

After parsing, an enum is output for every phantom type — and an extension of Path is output for every phantom type. Each extension contains members to access child nodes of the generated tree.

Future directions for automatic code generation

The current strategy is very basic and you could quickly run into issues that the current version of the script cannot account for:

Names of generated phantom types could clash with existing type names in your code. This could potentially be resolved by supplying a prefix or a postfix for the generated phantom types when you call the script.

Names of generated phantom types could clash with each other — if you use the same names for separate path elements in your structure. This could potentially be solved in the future by allowing you to describe desired generated type names along with the tree structure in the JSON file. We just need to think of a good way to describe this. (Suggestions are very welcome).

Currently you cannot model collections of collections (ie. two nested <id> elements). The only current workaround is to model any such parts of the database hierarchy by hand.

Suggestions for improvements — and suggestions for how to solve the annotation issues in the JSON format are very welcome.

The open sourced frameworks

The concepts discussed in the previous blog posts have been split into two frameworks:

This means that non-Rx users can still benefit from Codable support and also take advantage og the Path concepts.

RxSwift users on the other hand can just grab the other framework to get the full range of expressivity.

Both of the frameworks are built as statically linked frameworks. This is due to the fact that the Firebase framework suite is already distributed as statically linked frameworks.

One benefit from distributing the frameworks this way is that at runtime, you will not have to pay the extra cost of loading the libraries dynamically.

Usage

Please refer to the README.md of each framework for usage examples.

Installation

Currently the only installation method is through Carthage — please reach out if you are interested in CocoaPods support as well, or perhaps even support the project by creating a pull-request adding CocoaPods support! :-)

In this post, we are going to add support for RxSwift. If you want to learn more about RxSwift, please refer to the official github repo, which has excellent examples of use cases.

The only further motivation I will present here is that there is an huge overlap in the mindset of the Firebase Real-Time Database and Rx: the RTDB allows the user to subscribe to data as it changes over time. One of the main purposes of Rx is exactly to provide primitives to model data as it changes over time.

As I see it, Rx is the perfect way to consume the Firebase RTDB APIs. For instance, observing changes for a location in Firebase using the native APIs require the user to keep track of a subscription token that is returned when subscribing. In Rx we can wrap this token in the subscription so that the Firebase observer is removed when the subscription is disposed.

But all this is probably easier to grasp when we start coding!

Adding an Rx extension

Let us start by adding an Rx extension to a DatabaseQuery (which is a super type of DatabaseReference) for convenience. The pattern for adding Rx extensions can be found in the RxSwift repo — for instance in the RxCocoa extensions. It looks as follows:

For the wrapping of a single observation we return a Single<T> type from RxSwift. Single represents an observable that can have at most one element. This means that it either always completes (returning that single element) or fails with an error. In that respect, a Single can be compared to a Future or a Promise¹.

For the wrapping of continuous observations we return an Observable<DecodeResult<T>> . When creating an Observable you need to return a Disposable that will be used to keep track of the life time of a subscription to an Observable. The closure passed to Disposable.create will be called when a subscription is disposed, and this provides a convenient way to remove the actual Firebase observer.

You may wonder why we use different generic types T and DecodeResult<T> for the Single and the Observable respectively. The reason for this has to do with the way that Observables work. For the Observable case: as soon as an observable has an error, the signal ends. This means, that if we treated errors (like network errors, missing data errors) like Observable errors, then the signal would fail upon missing network. We might not be interested in such a behavior, so instead we let our signal contain a sequence of DecodeResult<T> — then the user has more freedom to interpret the errors as they see fit. We will add convenience functions to ‘unwrap’ the result type below. For the case of the Single we do not have the issue of the signal terminating after the first error, because the Single always terminates after the first value. So in this case there is no reason to keep the DecodeResult wrapper.

Extending the FirebaseService

Let us extend our FirebaseService from the last post to add support for type safe paths:

So here we replace all the observe functions to return an Observable (or a Single) instead of taking callback closures as we did in the previous post.

Filtering DecodeResults

If you like to live your life dangerously, or have no sound way to handle errors, you may wish to filter away the errors, or have them logged, while only actually handling the success cases where model entities are received and parsed successfully. For this purpose we could create overloads of the observe functions. But as the amount of wrapper functions could grow in the future, we could instead add a generic way of filtering Observable<DecodeResult<T>> signals.

This can be done by a constrained extension on Observable, but we cannot constrain on DecodeResult since this is itself a generic type. What we can do is create a protocol and use this to create our constrained extension:

To explain the code above: We add a protocol ResultProtocol to use for our constrained extension. Then we conform Result to this protocol. Now we can create the extension constrained to Observable types containing ResultProtocol-conforming elements.

One function, filtered(), simply throws away all error values in the Observable stream while filtered(handler:) gives you an opportunity to intercept error values — for instance for logging. If you have a Singleton’ish logging concept, you could use this to create your own small extension that filters success values and logs errors.

In the Swift Talk, Brandon Kase and Florian Kugler discuss `Phantom Types` which in short are types that are never instantiated, but only ever used as generic constraints. These can be a really powerful tool for adding strong typing to APIs that are usually not strongly typed — just like the paths used in Firebase RTDB References.

Please, please go and watch the awesome talk on objc.io!

In this post we are going to mix Phantom Types with actual Codable model types to create a really powerful API.

Sample code for this post can be found on the typesafe_paths branch of the github repo introduced in the first blog post:

At the root level we have a chatrooms path and a configuration path. Under chatrooms we have a number of keys of individual chat rooms. For each individual chat room we have a name and a messages path. Name could just be a String and under messages we have a number of keys — and for each of these keys we have some Message entity. The configuration path at the root contains some Configuration entity.

With the above structure in mind, now we can create yet another constrained extension on Root paths that lets us access all the children in the root of our tree:

The type of chatRoomsPath is now Path<ChatRooms> and configurationPath is Path<Configuration>.

You can imagine building an entire tree of paths like this, but we are still only getting started! :-)

Using the Path type

For now, the only thing we can do is create paths and render them.

This can of course be used in conjunction with a Reference in order to provide a String path. But we can do better.

At this point I would like to introduce an alterior motive with the wrapping of the Firebase RTDB APIs:

As we are building up more and more general concepts, we are also abstracting away from the actual Firebase APIs. This means that we are basically able to hide away the Firebase APIs and only use our own wrapper. This is extremely handy when it comes to testing: The tests do not need to know anything about Firebase — only our wrapped APIs. You can also imagine replacing the Firebase RTDB with something else entirely. For instance you could possibly transition to FireStore using the exact same wrapped API.

Let’s define a type to handle all Firebase RTDB observations — later on we can add a protocol defining the API for this for easy test mocking.

So the compiler directly tells you that you are providing a path to a Message but a value of type Configuration. This makes it easy to spot the error in the code.

As you may begin to see, this is indeed very powerful. The hierachy of Path types now defines a type safe ‘schema’ the tells the compiler where you may write different kinds of model objects in the database!

The observe function also improves greatly from being bound to the types of a path:

let path = Path().configurations.observeSingleEvent(of: .value, at: path) { result in // result is now inferred to be of type: // Result<Configuration, DecodeError>

}

This is another huge improvement to the API.

Again the Path hierarchy tells us what kind of data can reside at a specific path, so the compiler will try to decode that specific type — like the Configuration type above.

Again we will get an error if we try to call the function with a path to a type that is not Decodable.

Paths to collections of model entities

The example above highlights an issue with our API:

We still have to send the DataEventType to our API — and actually the specified event type contains some semantics about the use.

For instance the .value type returns data at a specific path while .childAdded returns data from child nodes of the path. This means that if we use a .childAdded type on a path to a Message, then we will not get called with actual message data, but rather when data below a message is added.

Wouldn’t it be awesome if we could model that some paths referred to collections of entities and others to the specific entity itself?

At Ka-ching we have tried a few ways of modelling this, but far the easiest way is to basically duplicate our Path type. We will name the copy Collection and nest it inside the Path type in order to avoid clashing with the standard library Collection type.

This means that a path to a collection of messages will be typed: Path<Message>.Collection.

So the Path.Collection type differs from Path in that it has a child() function that given a String key always returns a Path to a specific instance of it’s own generic type.

The Path type is extended with a fileprivate version of append that can return paths to collections.

With this addition we can do a lot of new things. First of all our Path hierarchy becomes a lot smaller since we do not have to explicitly model the ChatRooms and Messages collections — they are now available ‘for free’:

Taking advantage of our collection paths

Now we can extend our FirebaseService in a number of ways. First let us get rid of the DataEventType parameter in our existing observer functions. It doesn’t make sense to call them with child added/changed/removed.

Instead we will also create new overloads for observing on collection paths where these event types still make sense.

Finally we will also add an addValue that may only be used on collections.

Conclusion

Adding the quite simple Path type and a small service class allows you to build a hierarchy which becomes a type safe definition of the structure of your database.

With this hierachy in place you make the compiler ensure that you can’t accidentally place data in locations that are not intended. And when observing, the type of data that is being parsed is inferred by the compiler. This reduces boilerplate and is yet another safe-guard with respect to parsing the data from the database into the correct type.

We are so thrilled to get rid of any stringly typed data in our code base, so thank you very much objc.io for the great inspiration for using Phantom Types! :-)

Future enhancements

It would be nice to have a function that could observe an entire collection of entities.

This can be done in a few different ways: We could just add another observe overload that returned [String: T] results using the .value event type. This will, however cause the entire collection to be re-downloaded and parsed every time a single entity changes in the database. Still, it could be useful when you know that you have small collections. The dangers of using this should be communicated in the API name.

Another way would be for the observe function to keep track of a [String: T] dictionary, observing both .childAdded, .childChanged and .childRemoved and then calling the handler with the new Dictionary each time a child is manipulated. This is a bit better than the above, but we are also moving towards some higher level logic that is further away from the existing Firebase APIs. We will have a look at this in the next blog post about wrapping all this up using RxSwift!

Another thing that becomes apparent when you are modelling your Path hierachy is how much this looks like an actual schema defined by some piece of data. Wouldn’t it be awesome if we could generate this boilerplate code based on a JSON representation of the Firebase RTDB hierarchy? I think we need to explore this in a fourth post in this three-part series. :-)

Background

The Firebase Realtime Database is an excellent tool for building apps. There are plenty of articles covering some of the advantages of using Firebase, so this post will not go into details about that, but instead assume that you have some prior knowledge of the technology.

The iOS APIs for Firebase are currently written in Objective-C, and although the APIs present themselves nicely in Swift, this does mean that we cannot take advantage of Swift-only features like Codable support out-of-the-box.

This post (and the following few) will explore the possibility of adding nice and ‘Swifty’ extensions to the Realtime Database API.

Motivation

The motivation for this first post is to allow you to consider the data stored in the Realtime Database as model objects instead of as pure data.

For the second post we will get rid of the error prone use of String paths to data.

The third post is motivated by the fact that there is a pretty big overlap in how the world is perceived through Rx glasses and how the Realtime Database APIs work. Namely the fact that data is in both concepts percieved as values that change over time. Let us take advantage of this by combining the concepts.

TL;DR

The goal of this post is to be able to use Codable model types with the Firebase Realtime Database APIs like this:

Codable

Many other articles explain Swift’s Codable in greater detail. Suffice it to say here, that annotating your model types with Codable conformance provides an elegant and convenient way of allowing your types to be serialized into various formats.

One of these very common formats is naturally JSON. Support for decoding from and encoding to JSON is provided through the types JSONDecoder and JSONEncoder respectively.

Unfortunately, these two types consume and create JSON in the format of Data instances containing the String representations of the JSON — while the Firebase RTDB APIs create and consume JSON ‘structures’. Basically this means nested values of type Dictionary, Array and ‘primitive’ types like strings, numbers and booleans.

This means that there is a gap between the two APIs. One way of bridging this gap would be to take the Data output from JSONEncoder and deserializing it into a structure using the JSONSerialization API. But this doesn’t feel quite right since it basically needs you to do extra work.

Fortunately, Swift is Open Source, and the current implementation of JSONEncoder and JSONDecoder almost already does what we want.

So here is a small recipe for making your own Firebase RTDB-compatible Encoder and Decoder pair.

Cloning JSONEncoder.swift

Create a new project in Xcode. Let it be a Cocoa Touch Framework.

Find the JSONEncoder.swift file in the github repo for your version of Swift. In this example we’ll go with Swift 4.1

For the encode function, replace the return type Data with Any, remove the JSONSerialization step and return topLevel directly.

Similarly, rename JSONDecoder to StructureDecoder

In the decode function, replace the input from json: Data with from structure: Any and call _decode directly on the structure instead of deserializing the json data first.

Be sure to import Foundation in both the renamed StructureEncoder.swift and Codable.swift

Remember to add attribution to the apple/swift project in your library or app.

Now you have a functioning encoder / decoder pair that can be used together with the RTDB APIs!

Add the Result type to your project

This is of course based on your own preference, but there are good reasons for modelling the input to an asynchronous callback using a type like the popular Result-type implemented by antitypical — instead of as a pair of optional version of your model type, and an optional Error .

Conclusion

With just a little groundwork, we can now use the Firebase Realtime Database APIs together with Codable types. The APIs are still a bit clunky to use since you need to supply the generic type argument inside of the callback function signature.

We will deal with this bit of clunkyness later. Until then you can imagine the observeSingleEvent returning some kind of Future type, so that the type can be inferred by the assignment — or when Swift support for async / await lands (in Swift 6, perhaps?), observeSingleEvent could be an async function used like this (assuming that async functions are also throwing):

Considerations

The downside of using JSONEncoder.swift from the swift repo is that it needs to be maintained as swift evolves. There has been talks in the Swift forums about adding a StructureEncoder and StructureDecoder pair to Swift, but from the discussions it appears that they would not get JSON ‘semantics’, so likely no support for automatic encoding and decoding of keys to snake case and the like.

At our company we have maintained a version of the StructureEncoder.swift through 3 swift releases, and it has not been a very big deal to maintain it so far.

Github

We created a small repo to demonstrate the changes to JSONEncoder.swift and the extensions to the Firebase API.