A couple years ago Krupa filled up a
whiteboard with boxes and arrows, diagramming what the AMO systems looked
like. There was recently interest in reviving that diagram and seeing what the
Marketplace systems would look like in the same style so I sat down and drew the
diagrams below, one for the Marketplace and one for Payments.

Marketplace:

Payments:

Honestly, I appreciate the view, but I wince at first glance because of all the
duplication. It's supposed to be "services from the perspective of a single
service." Meaning, if the box is red, anything connected to it is what that box
talks to. Since the webheads talk to nearly everything it made sense to put
them in the middle, and the dotted lines simply connect duplicate services. I'm
unsure whether that's intuitive though, or if it would be easier to understand
if I simply had a single node for each service and drew lines all over the
diagram. I might try that next time, unless someone gives me a different idea. :)

Lastly, this is the diagram that came out first when I was trying to draw the
two above. It breaks the Marketplace down into layers which I like because we
emphasize being API driven frequently, but I'm not sure the significant vertical
alignment is clear unless you're already familiar with the project. I think
finding a way to use color here would be helpful - maybe as a background for
each "column."

Or maybe I'm being too hard on the diagrams. What would you change? Are there
other areas you'd like to see drawn out or maybe this same area but through a
different lens?

A common aspect amongst the regions Firefox OS targets is a lack of dependable
bandwidth. Mobile data (if available) can be slow and expensive, wi-fi
connections are rare, and in-home internet completely absent. With the lack of
regular or affordable connectivity, it’s easy for people to ignore device and
app updates and instead opt to focus on downloading their content.

In the current model, Firefox OS daily pings for system and app updates
and downloads the updates when available. Once the update has been
installed, the download is deleted from the device storage.

What if there was an alternative way to handle these numerous updates? Rather
than delete the downloads, they are saved on the device. Instead of each Firefox
OS device being required to download updates, the updates could be shared with
other Firefox OS devices. This Goodwill Update would make it easier for people
to get new features and important security fixes without having to rely on
internet connectivity.

Goodwill Update could either run in the background (assuming there is disk space
and battery life) or could be more user-facing presenting people with
notifications about the presence of updates or even showing how much money has
been saved by avoiding bandwidth charges. Perhaps they could even offer to buy
Bob a beer!

When we run ALTER statements on our big tables we have to plan ahead to keep
from breaking whatever service is using the database. In MySQL, many times* a
simple change to a column (say, from being a short varchar to being a text
field) can read-lock the entire table for however long it takes to make the
change. If you have a service using the table when you begin the query you'll
start eating into your downtime budget.

If you have a large enough site to have database slaves you'll have a
double-whammy - all reads will block on the master altering the table, and then,
by default, the change will be replicated out to your slaves and not only will
they read-lock the table while they alter it, but they will pause any further
replication until the change is done, potentially adding many more hours of
outdated data being returned to your service as the replication catches up.

The good news is, in some situations, we can take advantage of having database
slaves to keep the site at 100% uptime while we make time consuming changes to
the table structure. The notes below assume a single master with multiple
independent slaves (meaning, the slaves aren't replicating to each other).

Firstly, it should go without saying, but the client application needs to
gracefully handle both the existing structure and the anticipated structure.

When you're ready to begin, pull a slave out of rotation and run your alter
statement on it. When it completes, put the slave back into the cluster and let
it catch up on replication. Repeat those steps for each slave. Then failover
one of the slaves as a new master and pull the old master out of rotation and
run the alter statement on it. Once it has finished put it back in the cluster
as a slave. When the replication catches up you can promote it back to the
master and switch the temporary master back to a slave.

At this point you should have the modified table structure everywhere and be
back to your original cluster configuration.

Special thanks to Sheeri who explained how to do all
the above and saved us from temporarily incapacitating our service.

Jared, Stuart, and Andy recently spent some time focusing on one
of the Marketplace's biggest hurdles for new contributors: how do I get all
these moving pieces set up and talking to each other?

I haven't written a patch for the Marketplace in a while so I decided to see
what all the fuss was about. First up I, of course, read the installation
documentation. Ok, I skimmed it, but it looks pretty straight forward.

Step 1: Install Docker

I'm running Ubuntu so that's as easy as:

sudo apt-get install docker.io

To fix permissions (put your own username instead of clouserw):

sudo usermod -a -G docker clouserw

Step 2: Build Dependencies

The steps below had lots of output which I'll skip pasting here, but there were
no errors and it only took a few minutes to run.

Hmm...that doesn't seem good. Apparently there is some work left to get the
tests to pass. I'll file bug 1082183 and keep moving. I know Travis-CI
will automatically run all the tests on any pull request so any changes I make
will still be tested -- depending on the changes you make this might be enough.

I'll avoid talking about the patch since I'm trying to focus on the how and
not the what in this post. The code is all in the /trees/ subdirectory under
wharfie, so I'll go there to write my code. A summary of the commands:

Now my changes are on Github! When I load the repository I committed to in my
browser I see a big green button at the top asking if I want to make a pull
request.

I click the button and submit my pull request which notifies the
Marketplace developers that I'd like to merge the changes in. It will also
trigger the unit tests and notify me via email if any of them fail. Assuming
everything passes then I'm all done.

This flow is still a little rough around the edges, but for an adventurous
contributor it's certainly possible. It looks like Bug 1011198 - Get a
turn-key marketplace dev-env up and running is tracking progress on making
this easier so if you're interested in contributing feel free to follow along
and jump in when you're comfortable.