Designer, programmer, App Store incumbent

Today I took a break from work to watch the most recent YouTube video by my favorite Fortnite streamer, Tfue (yes, I like to watch people play video games). Anyone who watches Fortnite videos on YouTube knows that there are usually “V-Buck scams” in the comment section of every popular video. These are comments by scammers promising to hack some in-game currency for you so you can unlock items in the game without paying real money. They are typically phishing schemes or survey-based offer walls that earn the scammer small amounts of money. If the creator of the video doesn’t proactively delete these comments, they become pervasive.

Early V-Buck scams were pretty obviously sketchy to anyone who’s been on the internet for a while:

The thing is, the scammers didn’t have to try very hard. Fortnite’s main audience is kids and teens, and for those viewers, Fortnite content on YouTube might be some of the first video content that they’ve ever truly engaged with. They’re easy targets for someone looking to carry out a low-effort con.

Recently, the scams have become more intricate. The scam accounts, in a very meta move, now play on the fact that “V-Buck Scams” are a meme in the Fortnite community:

You might notice something else about this comment: It has tons of likes and replies. Fake likes from bot accounts give it an initial bump to make it the top comment, where it gets even more likes organically. Why would anyone like this comment, though? Well, look at the replies:

An entire army of shill accounts varying in name, comment style, and degree of agreeableness with the original comment show up to control the conversation. Since these replies were made quickly after the comment was posted, they show up first and drown out any replies from real users. You don’t see those until you click “Show more replies”:

These repliers noticed the scam, but they were also amazed at how well it was orchestrated. Frankly, I’m amazed, too. The social proof of the high number of likes and seemingly real replies lends immense credibility to the original comment.

If you saw that first batch of replies with no other context, would you be able to immediately say without a doubt that they were inauthentic? Do you think a young child or teenager would be able to step back and see the ruse, or would they capitulate and try to access the website that would lead them to Fortnite glory?

What if these types of fake comments weren’t on a video about Fortnite, but rather a political video? A news clip, a debate, an interview? What if, instead of promising free virtual currency, they were making false claims about a candidate, or promoting extremist or anti-democratic values?

If someone was willing to go through such effort to perpetuate an online gaming scam, how much effort would someone put in if they wanted to control the thoughts or opinions of an entire nation to sew divisiveness?

On a video about the recent attempted terrorist attack with mailed bombs. Are these comments real?

The 2018 US midterm election is just 9 days away. In the past year, Facebook and Twitter have responded with various measures to controversies about hate speech, fake accounts, political manipulation, echo chamber effects and more. It’s debatable how effective their measures have been, but I fear we’re missing the biggest offender in plain sight: YouTube.

YouTube’s reach is absolutely massive. As of mid-2018, it has 1.8 billion logged-in MAU (Monthly Active Users), compared to Facebook (2.2 billion) and Twitter (330 million). The 1.8 billion figure excludes hundreds of millions, possibly over a billion, monthly viewers who do not have accounts, making YouTube the most-used social media site in the world.

It’s also the preferred platform amongst US teens: According to Pew Research, 85% of teens say they use YouTube, compared to Instagram (72%), Facebook (51%) and Twitter (32%). YouTube is extremely popular with our most vulnerable, impressionable population.

I fear that YouTube comments are being used to control the conversation and manipulate voters of all ages on a massive scale, and the problem is not going away.

What is Google going to do about this, and how long will it take? Will we allow this to be swept under the rug as we all spend hours watching videos on a website where the comments have little to no chance of being provably authentic or organic? Do we trust ourselves to be able to know what’s real and what’s not, and when someone is trying to control the conversation?

Right now, it seems too easy for bad actors to get away with this. And right now is exactly when we need that not to be the case.

With the impending shutdown of the Parse hosted service today, I want to reflect on why I think Parse failed and what the service meant to me. As a solopreneur and 8+ year iOS developer who has used Parse in all of his major projects, I think I have a worthy perspective on the value and shortcomings of the Parse service.

Parse got me to where I am

I’ve used Parse in all three of my most popular projects: Slide (5m+ installs), Falcross (350k+) and Jake and Amir (25k+). I don’t know if I’d have finished any of them without it.

As a budding developer who knew nothing about databases and had limited server software/deployment skills, Parse was a godsend. Parse provided an accessible abstraction to the “put this data online somewhere” problem, and allowed me to get familiar with the fundamental concepts of data storage, push notifications, server-side code, error logging and more without having to spend my time and patience on making sure these things worked in the first place.

Parse allowed me to focus on the products I was creating and not worry about the backend, which made me feel powerful and inspired me to put as much effort into making my apps great as I possibly could.

The Facebook acquisition

In April 2013, after I’d been using Parse for about 9 months, it was acquired by Facebook in an apparent bid to start providing mobile app infrastructure services to developers. Whether this was the first nail in Parse’s coffin is up for debate, but at the time I was mostly excited that Parse had access to significantly greater human and capital resources.

What killed Parse?

Many people have analyzed the business reasons that Facebook may have shuttered Parse, but most analyses haven’t focused on the nitty gritty technical details, and I believe that technical shortcomings played a large role in the decision to nix the service.

In this post I will focus on how Parse failed as a technology product, a topic with which I am intimately familiar.

Parse Technology: The Good

Mobile SDKs: The Parse iOS SDK was generally iterated upon quickly and without breaking changes. It eventually gained the ability to use custom subclasses to represent Parse objects which was useful in an OO sense. It also gained the ability to store objects locally for offline use, which actually ended up as a disaster (see “The Ugly”).

Query System: The Parse.Query mechanism provided an abstraction of MongoDB for reading, writing an updating database records. The API was simple, consistent across platforms, and features like scoresQuery.include('user') allowed for easy joins across collections.

Parse Dashboard & Data Browser: The dashboard was the ultimate one-stop shop for managing your app. Browsing database collections in realtime, including sorting, filtering, and row-level editing, were all a breeze with the Data Browser. Other sections provided many useful metrics like API requests per second, DB usage, number of active installations, recent push notifications, server logs, and more. One of my favorites was Jobs, a lovely UI for scheduling repeating or one-off background tasks which still beats anything I’ve found on npm.

Cloud Code & CLI: With beforeSave, afterDelete, and other model hooks, it was easy to validate or alter your data, or perform additional actions when something changed. The CLI made it easy to get started with CloudCode and lightning fast to deploy it, and even had an interactive dev mode that would deploy your changes every time a file was saved. Magical.

Cloud Modules: There were many integrations for sending e-mail, content moderation, and more. Nowhere near the scope you could get from the npm registry, but just enough to be intriguing and approachable to a young developer who’s new to APIs.

Push Notifications: Parse mobile SDKs saved device tokens automatically with relation to the current user, so you could easily target some or all of a user’s devices. Push channels made it easy for users to opt-in to wide-reaching notifications about specific topics of interest.

Design & Documentation: The Parse website and dashboard felt “Premium” and conveyed a certain sense of order and seriousness. The docs, while not perfect, did a very good job of getting your integration to a working state without being confusing.

The Bad

Multi-User Development

One project I did involved multiple developers working on one Parse codebase. I won’t get into too many details, but let it suffice to say that it was really annoying to do. Each user basically had to create their own Parse project, manually keep it in sync with the latest git HEAD, commit and push changes, and then one person had to log back into the “production” app, make sure they pulled the right changes from git, and deploy via the CLI. This invariably lead to multiple people manually pushing to production.

I’d love to know if anyone made this work well, but from what I can tell, Parse was designed with a single developer in mind.

Service Limits

Aside from documented service limits, like 600 req/s and 1000 objects per query, Parse had some really crippling undocumented limits. There are multiplesources where the community has compiled these limits. Some of the most heinous ones include:

100 line log limit: Want to catch a rare error or log something in production? Better be quick! There’s no way to send these logs to a logging service like Papertrail.

Unknown server capacity: Want to do something simple like resize an image on the server? The SDK supports it, but it fails randomly due to limited resources.

Push notifications are not guaranteed to be instant: They can arrive after 30 minutes or an hour. Goodbye chat apps or anything time-sensitive.

Support Options

I was hoping the increase in resources from the Facebook acquisition would allow the team to tackle some rough edges that needed to be smoothed out, especially with support. That didn’t happen.

They eventually shut down the in-house support website in favor of telling people to submit to StackOverflow or the Facebook Bug Tracker, and they shut down their Enterprise tier which offered phone support and was the only guaranteed way to get in contact with a member of the Parse team.

Having no options to even pay for guaranteed support made Parse an immediate “no” for any serious decision maker in a startup.

I have to give a special shout-out to Héctor Ramos who seemingly single-handedly answered the flood of support questions and gave thoughtful answers for literally years.

The Ugly

Database Automation & ACLs

Parse abstracted away database management completely, giving you no control of indexes and instead relying on a completely automated system to create appropriate indexes based on query patterns. This made the product simple, but started to break down when large collections were queried with multiple patterns, causing tons of indexes to be automatically added.

The need for additional indexes was exacerbated by Parse’s multi-layered access control system, which added some combination of _wperm, _rperm and _acl keys to any custom object saved, and added _rperm to any queries for these objects. Because of MongoDB 2.x’s inability to use more than one index for sorted queries, every key you wanted to sort by needed a compound index with _rperm, not to mention the additional compound indexes required for sorting by multiple keys.

You can see the result of this below on my mLab cluster which contains the migrated Parse data for Falcross. The collection in question has 30 million objects with <10 small keys per object. These are the indexes required to support just a few simple query patterns without grinding the cluster to a halt:

This 13.4GB collection has a whopping 22.2GB of indexes and the degraded write performance that comes with them. I need to point out that the 7 indexes above were created by me, by hand, after migrating my Parse data. I canceled my initial migration which was slated to take days with the unbelievable 13 indexes that the automated Parse system had created on this same simple collection. What a waste!

This unfortunate scenario is just as much a shortcoming of MongoDB as it is of Parse. I’ve since migrated the above collection to PostgreSQL and it occupies less than 10GB of space including the 4 indexes that it needs. Lesson learned: don’t use a document store for relational data.

I’ve gained the sense that database ops are complicated for a reason and that creating a system to automatically optimize a database is really, really hard. It’s a challenge that fits the scope of a whole startup, not a feature in a BaaS startup.

I do wonder if some company will come along and create some middle ground of abstraction between, say, Parse and Heroku Postgres, where a relational database is mostly abstracted away, but you can at least add/remove columns and manage the indexes yourself.

Local Datastore

This highly anticipated feature was Parse’s solution to store app data for offline use. It launched in December 2014 on iOS and looked amazing on the surface.

Unfortunately, upon implementing the LDS, I noticed almost immediately that pinning more than 1000 objects made any offline queries unbearably slow and pegged my device at 100% CPU for multiple seconds. I submitted a bug report and was told that this was “by design”. I was pissed, as you can see in my fiery response to the Parse team.

This taught me a painful lesson about the danger of using a proprietary platform and was a major turning point for me. My future plans for Falcross pretty much all revolved around being able to store things offline, and Parse’s solution wasn’t even close to being good enough for use in production. I saw no way to move forward in the timeframe that was necessary to keep supporting myself with my app income, so I went and joined my friend’s company.

As I later learned when Parse open-sourced their SDKs, the reason it was so bad was because it fetched the entire SQLite table for whichever entity you were querying and then filtered the objects in-memory in a proprietary way. What the actual what?

Even today, 2 years later, the feature looks to be still broken and I doubt it will ever get fixed.

Conclusion

Parse tried to do a lot of things, and it succeeded at many of them. It was the gold standard of the “mobile backend as a service” product category and shaped the direction for similar services, but ultimately failed to create a product that was solid enough to rely on for any serious project.

It’s clear that the team’s inexperience with databases led them to make some irreversibly bad engineering decisions, like basing the whole product on MongoDB, or designing the Local Datastore the way they did. Many of the thousands of questions on StackOverflow are about the limitations of the query system.

After the Facebook acquisition, I think the Parse team knew that their platform couldn’t be used at scale without serious overhauls, and so they dropped the Enterprise option and focused on the independent/hobbyist dev market. Unfortunately, there’s no money in that–who would pay for a platform that is only technically capable of supporting side projects that will either never grow to scale or will necessitate a full rewrite to grow to scale?

Thank you, Parse team

Thank you to Ilya and whole Parse team for being brave enough to break new ground. Your creation allowed me to achieve dreams I never thought possible before I knew about Parse. Your design decisions have influenced my engineering practice and I know that many of the brilliant ideas from the Parse platform will live on in other products.

And a special thanks to Facebook for being an unrelenting supporter of open source software and doing a graceful shutdown of the service with a year’s notice. I’m sure the people who continue to use the Parse platform on their own servers are beyond grateful for the opportunity to continue their projects that they’ve invested so many hours in.

What’s next

There are hosted alternatives to Parse out there, and even some companies who will host your Parse Server to provide a “Parse-like” experience. But since I don’t know a lot about either of those things, I’ll leave it to the reader to figure out what’s the best option for her.

As for me, my year serving as CTO of Onerent taught me a ton about wrangling my own stack, and that’s what I’ll be doing from now on. I love the freedom of building my backend however I like, and I feel so much more capable with RDBMS and node than I ever did with Parse.

That being said, I hope that something comes along that will give young developers the same feeling of empowerment that I felt when I found Parse. There is a special educational value in working with a highly abstracted backend. Whether there’s any money to be made there is a toss-up.

I got a week’s supply of Soylent 1.4 a couple weeks ago. I planned to replace half of my meals in order to save time and improve nutrition. I started right away with half a pitcher per day, and I was loving it. I felt healthy and energetic and my IBS was notably improved. I also didn’t mind the taste.

On day 5, I got sick a few hours after drinking 1.5 glasses of Soylent. Crazy diarrhea and stomach discomfort, couldn’t sleep all night, and finally at 8:30am threw up for the second time in my adult life (I’m 22). I was burping a lot and most of the burps tasted like Soylent…not pleasant.

It took me 4 days to recover, surviving only on simple foods. My appetite wasn’t fully recovered for another couple of days. I have no idea if Soylent had anything to do with my sickness, and I won’t pretend like I do.

Now I’ve been recovered for a few days and the idea of drinking Soylent is absolutely repulsive. I thought I was being silly so I made a batch, but drinking it practically makes me gag. I don’t think I can ever drink Soylent again.

Thus concludes my (failed) Soylent experiment. I truly believed I could improve my overall health and save time without missing real food, but I was wrong. I learned that I actually really love food.

I see big issues with Soylent appealing to the mass market if they can’t make it more palatable and include different flavors out of the box.

Update 1/30/2017: I actually gave Soylent a second chance and tried 2.0 in late 2015, which became a staple of my diet for almost a year! I stopped drinking 2.0 because I felt like I was an increasingly severe allergic reaction to it, possibly related to soy.

One of the biggest challenges as an indie app developer is getting your app discovered. My game Falcross has been doing well on the App Store for over 2 years, but download growth is flat. I’m on a mission to change that, and to share my progress with other developers.

Falcross is a puzzle game where you use logic to uncover hidden images. You get 3 puzzle “packs” for free and buy the rest via IAP ($5 unlocks everything). There are also earn-able/purchasable “tokens” which you can use to get hints, revive when you lose, and more.

The IAPs perform well, with content unlocks and tokens selling at about an equal rate. That’s great news, because it means I have value to trade for users’ shares and invites.

Falcross had no share or invite features 3 months ago. I’ve added some in the last few releases, and this is how it’s going so far:

1. Tweet to Revive

When you make too many mistakes on a puzzle, you lose and need to restart from scratch. This can be a 15-20 minute endeavor on larger puzzles. The game offers you a “revive” for 15-25 tokens (~$0.50), which erases your mistakes so you can keep solving where you left off.

As of 4.7, you can also tweet to revive once per day.

The game generates a nice twitter-optimized image of your progress for you to tweet:

The blue shows how many people saw the tweet preview and dismissed it, and the red shows how many people continued on to tweet.

Hmm. 10-20 successful tweets per day. Let’s see how that compares to paid revives using tokens:

See that little green sliver at the top? Yeah, me either. 10-20 tweets per day and ~400 daily revives means that only about 4% of revives came from a tweet. In other words, users would rather spend a bunch of tokens than send a tweet for free 96% of the time.(Note: This stat could be a bit deflated, as tweet to revive is only available once per day and token revive is always available)

Conclusion: 10-20 tweets per day isn’t necessarily awful, but I’m barely getting clicks to my bit.ly link from Twitter. That’s because most of the people who tweet to revive only have a few followers, and they do it every day. This channel needs work.

2. Invite Friends via Profile

The Invite Friends button is shown to logged in users, who account for about 6% of DAU (~250 logged in users per day). Here’s a daily graph of invites sent from this screen:

Not a lot of activity, but these invites are non-incentivized. Interestingly, Messages is the most popular way to share invites, followed by Twitter:

Conclusion: Not bad for a non-incentivized channel, but overall has generated <10 clicks to my bit.ly link in a month. Results are likely skewed by the inclusion of an iOS app banner on the shared invite page–I can’t track clicks on that. I want to replace the invite page with a Branch deep link which will handle following the inviter automatically without requiring the invitee to return to the invite page. This sharing option also needs to be expanded to anonymous users, who account for 94% of DAU.

3. Share to Unlock All

What if, after completing 50 puzzles, you could just share the game to your Facebook or Twitter feed and unlock every puzzle pack for free ($5 value)? This is absolutely the craziest viral strategy I’ve tried, and it works. Targeted at new users only, it’s the most effective sharing incentive I’ve found yet:

This channel generates about 10-20 shares per day, all from unique users. It’s also generated over 60 clicks so far. I don’t have referrer data for those clicks, but I believe that a lot of them come from Facebook, as I’ve heavily optimized the shared link for Open Graph:

So that means lots of people are sharing to Facebook, right? Wrong.

I used a UIActivityViewController to handle sharing here, with all activities except Facebook and Twitter excluded. Unfortunately, that doesn’t exclude iOS 8 extensions, so people have been “sharing” to Flipboard, Evernote, and Pinterest and still being rewarded.

Conclusion: This channel works really well, but it needs to be limited to Facebook. If users are getting a $5 IAP for free, they better earn it by sharing in a way that will generate clicks! I’m excited to see what happens once I enforce this.

Interestingly, this promo doesn’t seem to be impacting IAP sales much, but I’m still A/B testing with Leanplum to find the optimal time to show the offer.

Next Steps

All the charts above are from Localytics, my favorite analytics tool. It’s very helpful, but I also need to be able to track clicks and installs, both of which are external to Localytics.

My friend showed me Branch, a free service which claims to track referrals from share all the way to install. Organic referral tracking is the key to quantifying my viral growth efforts, so Branch is absolutely going into the next version of Falcross. I can’t wait to see what kind of data I uncover.

I recently took up a side project that I’ve been meaning to pursue for a long time, and got some amazing results.

I’ve been a huge fan of Jake and Amir since high school. Unfortunately it’s hard to keep up with their videos because their website has poor mobile support and CollegeHumor, their parent site, is buggier every month.

Last month I decided I was fed up, and I knew J&A’s loyal fan base was fed up too. In our mobile-first world, it should be more than easy to view our favorite content on our phones and tablets. So I took matters into my own hands. I decided to develop my own iOS app for Jake and Amir.

The app itself I designed very minimally, with lots of iOS 7 standard UI and a bit of spark where I knew fans would appreciate it:

The app’s Random Video screen mocks Tinder (left). Allusion to a running joke on Jake & Amir’s podcast, If I Were You.

After a month of development and a 2-day review from Apple, the app was released on Dec 31st. I posted about it on Facbeook and the J&A subreddit, an active community with thousands of fans. The response was overwhelmingly positive: The app got 224 downloads and six five-star reviews in the first 24 hours!

Then yesterday, something magical happened:

Boom. 300 downloads in an hour. 1,325 by the end of the day. Tons of grateful comments from fans. And, best of all, a personal plug from two guys I really look up to!

I’m not making any money from the J&A app, but it’s one of the most satisfying projects I’ve ever done. As developers we have the ability to reach in and fill voids that nobody else is willing to fill, and that’s really powerful.

Fellow web and mobile developers, I’m calling on you to do one project like this per year. Find a person or company you really care about and write some software that will increase their exposure or make their life a little bit easier. Do something that they’d have to pay $10,000 for elsewhere, just because you can, and because you care. Wost-case scenario, you make the world a little bit better, and best-case, you get some exposure and connect with someone who would not have otherwise noticed you.

A message to CollegeHumor if you’re reading this: I’d love to work with you to implement your ads in the app so you’re not losing ad revenue. Please contact me at arman@rawrmaan.com!

In January, I started working as an intern for Astrid (recently acquired by Yahoo). Thanks to the kindness of the Astrid team, I was working directly on Astrid for iOS from my 2nd week on. I wrote shipping code as a college freshman.

In April, my game Slide shot to #1 Free on the US App Store charts and stayed there for almost 4 days. Almost 4 million downloads in a week. Whoa. How did that happen? Better launch Falcross.

In July, I was hired as an engineering intern for Apple on the iPhoto team. I worked at the main campus in the iconic main building, IL1. I did stuff I can’t talk about.

In December, I released the 1.4 update to Falcross and finally realized that I was capable of making a profitable game. Slide is not very profitable.

How I came into all of this fortune remains a mystery. Maybe people liked my design sense or my attitude. Well, probably. But why me? And more importantly, how long could this continue? Certainly 2013 couldn’t be nearly as prosperous as 2012. Or could it?

Fast forward to now. I feel like a sitting duck. I feel unproductive. Uninspired. But not because I became rich in 2012 and now I’m lazy. I didn’t become rich in 2012, at least not money-wise. I simply feel like I haven’t done anything notable this year. Of course, compared to last year, notability is a high calling. But I feel behind; I feel like I could have done more by now. Falcross 2.0 has been in the works for 5 months and it’s definitely getting there. But “getting there” and “shipped” are two very different things. I feel like it should have been done 3 months ago.

Maybe it has something to do with the way I choose to socialize. I got my medical marijuana card in December 2012 and I’ve certainly been wearing it out. Winter quarter I made a bunch of stoner friends, and as a result, winter quarter was spent in a smoky haze. Certainly I had some of the most fun times of my life, and spent it with lovely people, but I knew it couldn’t continue forever.

Living in fantasy land can be fun, and productive, but not for an extended period of time. Certainly I had some of my best ideas for Falcross when I was high, and surprisingly wrote a lot of good code. But I slowly learned that smoking weed 2-3 times a day is not conducive to being a productive person. Ideas start to lose meaning when you can’t even remember what you were doing 3 hours ago.

So I think it’s about time for me to hit the brakes. Hard. My past 4 weeks have born some of the least productive and inspired days in recent (hazy) memory. My sense of progress has become uncalibrated. My standards have fallen.

This is not a declaration of abstinence or a cry for help. It is simply a realization that I can’t become who I want to become if I continue my current habit. I will not stop smoking weed altogether because I believe there’s a lot of positive and introspective value to it. But my days of being a daily, twice-daily, thrice-daily smoker are over. This is my public commitment to myself.