Swift on the Server

Chris Bailey at Playgrounds Conference, 2017

playgroundscon

Video transcript:

Hey, good morning.

So as has just been mentioned, I'm Chris Bailey, and I work for IBM as part of the Swift at IBM Engineering Team. Before that, I spent 15 years working on Java, JVM’s and the Java language. I spent a couple of years working on Node.js and then I came to Swift.

And it's been a very exciting journey over the last, really only 15 months, since Swift was open sourced. And in that time, we've gone from having something that barely worked on Linux to the point that we now have, full production ready backends for Mobile applications written in Swift. And that's what I'm going to talk a little bit about today.

So, as has been mentioned, the topic of this presentation is about how Swift + Mobile = BFF. Now I'm not a teenager, so to me BFF means backend for frontend, apparently there's some other meaning! So I'll get rid of that, and that's what it is for me. So Swift + Mobile = backend for frontend.

Now this isn't an IBM thing. What I'm going to be talking about is this design pattern, where you have backend for frontends serving up your mobile experience with a separate backend for Web. Now this is a design pattern that was invented by a consultancy group called ThoughtWorks, and they did it doing consultancy for SoundCloud. So the concept of a backend for frontend, and having multiple backend for frontends depending on what frontend is, is something that was developed and is in production use at SoundCloud. And in fact, ThoughtWorks went on to extend the idea and reuse it at a company that's a bit more local. So realestate.com.au also uses the backend for frontend pattern that's been well documented by ThoughtWorks.

So what they did was, they went from this topology where they had a single backend which they call a monolith, and all of their frontends whether that's Android, whether that's iOS, Web, or some kind of business to business connection; was using exactly the same backend. As this is what they moved to, which I'll admit looks a lot more complicated but actually allows you to be a lot more agile in terms of delivering features, because feature teams become self-contained for both frontend and backend. So we'll talk about how to get to that point.

So, talking of topologies, let's say you have end users and they're using iOS, Android, or Web, and they are in the public network and you have a gateway to your data system, and then at the backend you have some kind of service that you want to use, whether you want to connect to Twitter, whether you want to do cognitive or use a database. So, in between, you're going to have some kind of backend API that allows your frontend devices to access backend hosted services. Now, in a normal situation, you're talking to that backend API directly from the client devices, and that's working with your backend systems. So you don't have to put things like JDPC data drivers inside client devices, and you don't have to store passwords for your backend systems inside your device application.

So that's where a lot of people are today. And as companies grow, they tend to go to slightly more complicated topologies. So you end up with a backend API, but you also have a bunch of services that may do things like logon or authentication, and so on. So more complicated versions are just converting services to microservices, but it's all fairly similar.

Now the way this journey works is you have an API Team that is responsible for the backend server, and you have a Web team, you have an Android team, and you have an iOS team. So who actually looks like this, where they work? Where ... having separate teams doing the frontend and doing the backend behind it? Okay. So that's actually almost everyone.

So the problem with this is first of all, that Web and Mobile tend to have some different requirements. So on Web, you care about First Paint Time. So the fact that you load something with a browser, until that data is downloaded from the server to the browser, you have nothing, right? You don't have a wire framework of content because there's nothing in the browser, it all has be loaded from the server. With a Mobile application, you've actually already got a lot of the UI display, the moment you open the application. So you've got a lot of pre-rendered content. So Web cares a lot more about First Paint Time. They care about servers like rendering as a way of accelerating that. So the server is actually taking dynamic code and converting that into HTML wire frames, to get that down to the client as quickly as possible so that they've got something that shows up very, very fast in the browser.

And they care about search engine optimization. So they care about how to make sure Google or one of the other search engines, you know not Yahoo any more, obviously, has the ability to actually index to the page. On Mobile devices we care about things like request frequency and network payload size because we're on a 3G or 4G network, where as a Web browser typically is on a stable internet connection. And we care about resource usage, things like CPU and battery and memory, that's being used, is a real impact to the user and the device. So the requirements of the two sides are actually very, very different.

So the requirements are different, and that brings me to an example that was given at the Swift Summit by someone called Sommer Panage. So she talked about Star Wars. Not really the film so much, but about APIs. So if you make a REST call, this is all you do, right, it's very, very simple; you make a small packet request of a server and it returns to a payload. So if it was for a To-Do list, I could ask it for one of my items in the To-Do list and it would return me a JSON of an object, with a title, a priority, and whether or not it is completed.

Now the problem with this, and this is where the Star Wars API comes in, the Star Wars API gives you information about every single Star Wars film, character, ship, etc. Now if I was to make a request for Person ID “1”, I get back Luke Skywalker. But if I want to know what starships he uses, it just gives me indexes to other REST calls I need to make. So if I want to know about Luke Skywalker and the starships that he's piloted, I have to make a call to this, and then I have to make calls to those two. So, once I've done that, I get another payload which is about the first starship, which is the X-Wing, and the second one which is an Imperial Shuttle. So that's quite a lot of data I've got back in three REST calls I had to do in order to find out what Luke Skywalker flew, in the films that he's been in.

So the point that Sommer Panage was making, was there's GraphQL. And GraphQL is designed such that you make a query that says, this is the data that I want. So rather than making three REST calls, you'd make a single call that specified that data that you wanted to get back. And this is a huge saving over having to get three lots of data returned from three REST calls. So you're saving a lot of network traffic. The problem with this of course, is every request has to describe everything that I need. So rather than making a simple REST call which contains very little data, I've got to make quite a large request. If I did this with a bespoke backend for frontend, that was capable of making those REST calls itself and filtering the response and just sending me back the data that I needed, I would only need to do this. I could set up my own backend for frontend, make a request to Person One, and it would just return what I need.

So this is making the case that rather than having this single backend API, I should actually have two or three of them. One for Web. And potentially one for Android and iOS. Or one for each. And that's because Android and iOS don't have all of the same requirements. You have different ways of doing push notifications. You have different release cycles. So for example when the latest version of iOS comes out and I want to exploit new APIs and I need a new server component for that, I want to be able to change my server immediately, and not be blocked on the fact that they have to do some changes for Android or they're all on a different release cycle. So this makes the case for having three backends, one per frontend, and they become your backend for frontend, for your frontend device. But they also become Best Friends Forever, because they are linked together.

So the second part of this is about Mars. So this is an example I talked about at the Swift Summit. So, in 19-, not 2024 which is when Elon Musk thinks that we're going to go to Mars, and now that Chris Lattner works for Elon Musk maybe Swift will be involved in that ... but back in 1998 there was something called the Mars Climate Orbiter. Now the Mars Climate Orbiter launched form Cape Canaveral with the aim of putting one of these in orbit around Mars to look at the climate, but also act as a satellite for coordinating future missions to Mars. Now unfortunately just about 300 days later, in 1999, in September 23rd, they lost contact with the Mars Orbiter that had been going for about 300 days to get there. And unfortunately two days later it was declared a complete loss.

Now what happened was, as the Orbiter was moving to Mars, it has a planned trajectory. And that initial planned trajectory is actually designed to send it to hit Mars, but shortly before reaching Mars it goes through what's called 'TCM4', which is trajectory correction manoeuvre. So, basically, it does a thrust burn to change its orbit and that's designed to bring it into orbit around Mars at a height of 226 kilometres. Now what actually happened was it came in a lot lower, did the same correction manoeuvre, but unfortunately came in 57 kilometres above the surface of Mars which is a little too close; leading to this. So a 300 million dollar failure occurred sending a probe to Mars.

Now the reason for that, is to do with this. There were two teams involved in creating the thrust calculations and thrust software for the Mars Climate Orbiter. Now one of them, Lockheed Martin based in Bethesda, did the ground software. The other team, at the NASA JPL in Pasadena, they did the trajectory calculation software. Now the two of them did what you expect for software engineering. They had a well-defined, what they called a SIS, a Software Interface Specification; so think of that an API between the two parts. And they were talking about total impulse, so one of the them had to calculate how much impulse, how much burn was required to do the manoeuvre, and the other half did the equivalent calculation for how long to actually burn the engines.

Now the problem is Lockheed Martin used pounds seconds, which is a US unit, whereas NASA was using newton seconds, the standard metric unit. And the problem is that one pound second is actually 1.488 newton seconds. So we had this huge problem that whilst we had a well-defined specification, and they did a lot of testing, and these are guys whose entire job is to be able to do things remotely, right? They are also rocket scientists, so this is supposed to be easy stuff for them. Unfortunately, whilst they both agreed they were sending total impulse, and they both agreed that was going to be a double, or a long, they had this mismatch on what that number actually meant, leading to a huge failure.

So what does that mean for our pattern? What it means is, rather than having this set of silos where you have an API team and you have a bunch of frontend teams, what you really should be doing is getting rid of that API team and making each of the frontend teams responsible for their own backend. Now, this, again, does cause some problems because it means that a Web team that does JavaScript and HTML, has to learn another language. Although, actually, they don't if they use Node.js, so then they're using JavaScript on both client and server. Likewise for the Android team. They're writing Java code on the Android device, so they can have a Java-based backend for their frontend. And the iOS team, well they can be running Swift.

So you can now actually have self-contained teams working in a single language doing the frontend and the backend for frontend, for their device. And being very, very agile in that if I need to make a change, it doesn't matter whether it's server or client, I am responsible for doing that, and I can deliver it on the timescale for the features that I need to deliver to my client. And, yes, there's shared business logic and there's shared things which can be moved into the services, so you're not duplicating code in the backend for frontend; anything that would be done in all three, that can pushed to a backend service and shared amongst them. But anything that interacts with the device and it helps you deliver user features, can be done inside the BFF.

So how do we build a BFF in Swift? So there's a number of Swift server frameworks that exist today. I work for the team that creates Kitura, but there's also Vapor and Zewo and Perfect and a number of others. And they're all doing fairly similar things and work in slightly different ways, so it's usually a case of choosing which one you actually prefer once you start trying them out.

So, in terms of Kitura, how you create an application? So first of all, you create a folder. Then inside that folder you initialize your package, and then you create a Package.swift. So everything that works on server-side uses Swift Package Manager. So you create a Swift Package Manager package description, and you include a dependency on Kitura. Then all you have to do, in the sources main file, is initialize the server. And this is pretty straightforward; you import Kitura; you create a router, and a router basically is the URLs that you're going to serve information on. Router.get, that means if you get a GET request, an HTTP GET, on that URL, which is just slash.

I'm going to send a response of "Hello world." I then create an HTTP server. The reason that's there is you can have many HTTP servers on many different ports, if you want to. So you can serve on port 8080, 9080, as many as you actually like. So you give it a port number, you pass it the router, and you tell it to run. And that's it. That is now capable of running a simple Hello World application on HTTP GET requests of slash, in about eight lines of code.

So then you just need to deploy it. So you do `swift build`. Then you run the result of the Swift build. And then you stick your browser to port 8090, and you've got a fully working Web server. So anybody can do this in under four minutes. We actually tested this on a bunch of managers, we thought those would be lowest common denominator for developers, and that was the results that we got.

So the problem is that really is kind of a 'toy' application, and you need to do more significant things than that. Now we've done a lot of talking about server-side Swift and why it's so good in terms of performance, right? We've got numbers that says it's like six times faster than Ruby. It's already faster than Node.js and Express. And I had a couple of conversations with people and they said, "Well that's interesting, but in all honesty hosting costs and the performance of my backends is not really a problem. If I'm deploying Ruby, it may be four times slower but the performance is easily enough for my users. What I care about is how easy it is to build and deploy and run my application. So the faster I can build it the better."

So we tried that ... as I said, we're letting you build a simple server in eight lines of code. But we know that you need to do more significant things than that, and you need to do it easily. So that's why we've started to create something called the Swift Server Generator, which is an accelerator for building applications.

Right. So let's make this a little bit bigger. Right, can everybody see that? Is that okay? Okay. So just to prove there's no smoke and mirrors, there is nothing in my directory, there's nothing up my sleeves! It doesn't mean I know how to type. Okay.

So we have a utility called Swift Server Generator, which helps you build an application. So if I wanted to build an application called Playgrounds, I can select it and I want to say that I'm going to store some data in Cloudant. Through coding we have support for many, many databases. So whether you use Mongo or Oracle database or Apache Cassandra or Redis or Postgres or MySQL, those are all supported, but at the moment in our builder we only support in-memory data stores and Cloudant, but we are extending that. So I can get it to set up a ... actually let's skip back and say I'm going to use memory, because I'll run it on my laptop. Yes, 'user memory data store'.

So we now also have the ability to do monitoring and data collection for Swift.So this will automatically insert a package called Swift Metrics for you. And I can chose to actually pre-package my application, so it can be deployed to a Cloud. And the only one we support at the moment is Bluemix, but again we're going to extend that to cover other clouds. And Bluemix has the ability to do auto-scaling. So this is ... when my application starts to struggle because it's hitting as much load as it can take, it will automatically add another instance and load balance between the two. Likewise, if it decides that the load is now dropped off the system and you don't need the second one, it will remove it for you automatically.

So I can build support for that. And this is now building me a whole bunch of files for my application, and it's going to start using Swift Package Manager to download all the packages and build it. So whilst that's going on, I will switch windows. Let's check this is the same place. Yep. Okay. That should be it. Okay. So we'll just have a look at some of the files we've created. So we've created a Package.swift file. Let's expand that.

So it's created my application called Playgrounds, and it's introduced a few files for us. So it's added Kitura. It's added logging for us, so it's put a whole number of log statements into the code so we know when errors are happening. It's added Swift Metrics which is our application monitoring. And it's added a configuration utility. Now what that configuration utility does is, it allows you to set things like the port and the host name and whether ... what level of logging you've got through a configuration file, rather than being hard coded. It also means when you deploy it to a Cloud, if the Cloud has a way of supplying configuration, it will automatically pick those up. So it's designed to implement a whole load of coding best practices for you, as part of the application.

We've also added a .gitignore file, so this is designed so that's it's ready to be pushed to GitHub. And the .gitignore file will make sure that you don't push your build artefacts and the packages which will get pulled down at build time; so that's set up for you. We have the config file that I mentioned. We have a README that's ready for you to get going with. And we have these things, which are a bunch of artefacts which means that if you wanted to push it to Bluemix and I said I needed Cloudant, it actually provisions a Cloudant database for you as part of the upload to the cloud.

So the aim is for it to build a whole lot of this stuff for you. Now if we actually look at the source code, we have our main.swift, so this is creating the application that we asked for, it's added logging. And all of the code is there so it's just an accelerator, you can go and edit the code as much as you like. It will ... you can then change it, run it, re-push it. And in terms of our generated app, we also have a whole bunch of generated code. So this gives us a router.get, which says if there's an HTTP GET request on this URL, we'll serve something called the swagger.yml, which we'll get to in a second. So hopefully this isn't going to take too much longer, because it looks like it's compiling.

Now once that application's compiled, I can actually run it locally and it will all work. But we haven't really told it to do anything yet. So it's given me a blank application that has logging and it has monitoring, and it serves this thing called the swagger.yml, but it doesn't really do anything else. So one of the other things that we can do with it, is get it to ... oops, there we go, is get it to implement a data persistence. So if you have a data model that you want to be able to store inside the database, you can actually specify what that data model is, and it will create a bunch of REST APIs for you to actually persist that data; so create a CRUD application; create, read, update, delete.

So if I give it a model name of To-Do, because I'm going to build a To-Do List, and I give it a property name of Title. So every to-do item is going to have a title. And I'm going to say that's going to be a string. And it's not going to be required and it's not going to be a default. And then I'm going to have a second property, which is 'completed', so this is just whether not I've completed my item. So because that's true or false I'll make it a Boolean, I'm going to make it, not required, but I am going to give it a default, which is 'false'. So I'm going to say that every item I create is not completed. And that's it.

So that's now created a couple of extra artefacts for us. And one of the key one's is that we now have ... if I refresh this, where are we? We now have in our definitions, a playgound.yml file. And this what's being served up on that REST route that I said that we had, an Explorer and a yaml file; and that's this. And this is called a swagger definition, which is kind of your API definition for REST calls. So this defines that To-Do model that I talked ... that we just created. And it defines the GET ... the POST, the 'delete' REST calls for it. Now this is useful because if you want to do some testing, or you want to be able to hand that API to another developer, it actually gives you a full definition of what your application does. So if I quickly import ... that's the right place, yep, my playgrounds.yml. And this, because I had that file, has full definition of everything my application can do.

So if I make this a little bit bigger. So if I decided to do a GET and we- first of all I should start my server. Okay. And that is running on 8090. Okay. So if I do a request to GET all of my To-Do items, I get an empty array back which is what you'd expect because we haven't actually put anything in there yet. So if I do a POST, again to localhost, 9080, and I decided to actually 'create adjacent object'. So we said that you had to have ... we're going to give it a title which is 'do demo', and that should ... oh, 'could not get a response'.

Try using 8090- 8090, thank you. And this proves that it's live!

Okay. Uh, we're doing a POST, we're doing a 'send'. Let's hope that's worked. Let's see if we find anything. Oh, I've got some empty arrays, this is not looking good. The demo gods are not smiling on me! Well let's try something else. Let's see if we've actually got the server running? Yep, the server's there. And let's have a look at the monitoring. So the monitoring tells me that my application's running and that I'm making calls. So certainly, I now have fully working monitoring, and I have a fully working server, now I just have to work out why my REST calls aren't working. Uh, let's try adding a second model to it. Let's go for, To-Do 2. Okay. Title, string required. Let's say that is required, with no default. And ... uh, required, no, default, yes, which is false. Okay let's try that. Let's bin that and do ... don't save. Don't save. Import. Okay let's import the new one, with To-Do and To-Do 2. Right. Let's go back and do our GET. 8090, yeah!

Thank you. Okay. Yep. That's working. Let's do a POST. 8090.

Okay that's better. That's giving me a full ID. And it's giving me that result when I do a GET. Okay so we have that running now. So not entirely sure what happened the first time, but there we go. So if I just quickly run back to the presentation and hit 'play'.

So, yes, we've been building accelerators, and the idea is to build out that code for you as quickly as possible to make it easier for you to get up and running writing a first backend application. So the aim is that everyone can become a Swift Stack full end-to-end developer.

We have all of our content on GitHub in IBM/Swift. Everything is documented on Kitura.io. The new monitoring capability is there and fully documented, and we're fully available on Slack.