One of the things that immediately appealed to me about Phoenix coming from
Rails its views. Once I started
creating functions in my views that returned HTML, however, I was instantly
confused about how to test them.

I was making a function in my LayoutView that rendered a link for a site’s
navigation. It included some css framework boilerplate stuff and would also add
the is-active class if the current path matched the link’s path. I wanted to
write a test like this:

# the LayoutViewdefmoduleMyApp.LayoutViewdouseMyApp.Web,:viewdefnav_link(%{request_path:request_path},text,path)dolinktext,to:path,class:nav_link_class(request_path,path)enddefpnav_link_class(path,path),do:"nav-item is-active"defpnav_link_class(_,_),do:"nav-item"end# the testsdefmoduleMyApp.LayoutViewTestdouseMyApp.ConnCase,async:truealiasMyApp.LayoutViewtest"nav_link is not active when not on that page"dolink=LayoutView.nav_link(%{request_path:"/other_path"},"Link text","/path")refuteString.match?link,~r/is-active/endtest"nav_link is active when on that page"dolink=LayoutView.nav_link(%{request_path:"/path"},"Link text","/path")assertString.match?link,~r/is-active/endend

If you try to run these tests, though, you will be surprised. The reason is that
the link function does not return a string. Instead, it will return something
like:

This is a tuple marked as :safe (meaning it has already been html escaped)
with the second element being an IO list. You can read about why it is an IO
list instead of a string in this interesting post by Big Nerd
Ranch.

So, how should this be tested? You could try to match on the IO list, but I
think it is a lot easier to reason about tests against an html string. I looked
through the tests for Phoenix.HTML on
github and came across the
Phoenix.HTML.safe_to_string/1 function. This takes a tuple with :safe as its
first element and an IO list as its second one and returns a string of html.
This is exactly what we were hoping for. Use that in your tests and they will
work as expected:

defmoduleMyApp.LayoutViewTestdouseMyApp.ConnCase,async:truealiasMyApp.LayoutViewimportPhoenix.HTML,only:[safe_to_string:1]test"nav_link is not active when not on that page"dolink=LayoutView.nav_link(%{request_path:"/other_path"},"Link text","/path")|>safe_to_string()refuteString.match?link,~r/is-active/endtest"nav_link is active when on that page"dolink=LayoutView.nav_link(%{request_path:"/path"},"Link text","/path")|>safe_to_string()assertString.match?link,~r/is-active/endend

I hope this helps you write tests for your views in Phoenix apps. I can tell you
from experience that if you let too much of your view code go untested (or let a
bunch of logic sneak into your templates) you will be scared to make sweeping
changes to your view-related code — so get testing!

What is a scoped find?

In the Rails world, a scoped find looks like this:

defedit@post=current_user.posts.find(params[:id])end

This is a useful security technique that either finds a post that belongs to the
currently signed in user or raises an ActiveRecord::RecordNotFound error
(which Rails turns into a 404). Doing finds like this prevents unauthorized
access without a bunch of boilerplate authorization code.

How can I do a scoped find in Ecto?

Since Elixir isn’t object oriented, you can’t do current_user.posts.find,
but you can do something similar. The Phoenix generation tasks use
Repo.get! which works
similarly to Active Record’s find. For those times when you need to more than
an id lookup, Ecto also has
Repo.get_by! which is
works like get! but with additional parameters.

It could be used liked this for the same functionality as the Rails/Ruby code above:

Streams in Elixir are
lazy enumerables. You can create a series of transformations that aren’t
actually run until you either call Stream.run(stream) or an Enum function
on it. This means a stream is a powerful interface to pass to calling code since
they can not only add additional stream transformations but can easily control
when the stream is evaluated.

The most flexible way to generate a stream is using
Stream.resource.
Its signature is (start_fun, next_fun, after_fun) and it outputs a stream.
This signature can be confusing so let me explain. Most of the meat of
Stream.resource is in the next_fun so let’s start by talking about that.

next_fun takes what the docs call “acc” (meaning accumulator) and should
return a tuple containing the next element in the stream (which should
inexplicably be a single element in a list) and the accumulator to be passed to
the next call of next_fun. We’ll see some examples of next_fun in a bit.

start_fun is a function that takes no arguments and should return the
accumulator for the first call of next_fun.

after_fun is a function that is called once the stream is done. It is meant to
clean up any open resources (for example closing an open file that was being
used in the stream).

Let’s look at some examples. First, we can create a stream that will produce
each prime number, from two to infinity:

The start_fun just sets up an empty list as the accumulator and the
after_fun does nothing (since we have nothing to clean up). The next_fun is
extracted as next_prime to take better advantage of pattern matching.

First off, if we have nothing in the accumulator, it just returns {[2],[2]}
which gives 2 as the first element and sets [2] as the accumulator. We don’t
set [2] as the accumulator in the start_fun because then 2 won’t be
returned as the first prime.

If we have elements in the accumulator, we keep adding 1 to the last prime
we’ve found until we find a number that isn’t divisible by any of the previous
primes. That means it is a prime number so we return it plus add it to the
accumulator as an additional prime.

The next_fibonacci function only returns the previous two fibonacci
numbers as the “accumulator” because those are the only two that are needed to
calculate the next one. I think it might be better to think of the “accumulator”
as the context of previously generated results, regardless of whether it
accumulates or not.

Generating an infinite number of fibonacci numbers or primes is all well and good, but how can we
actually use Stream.resource to do something real?

This module exposes an async_map function that is meant to work like
Enum.map but to run the function passed to map asynchronously in parallel. It
then sends each item to the stream as its command finishes so the whole list is
processed in the amount of time it takes for the slowest one to finish. I use it
to make many http requests at once.

One thing to note is that instead of returning {[next],acc} from next_fun
returning {:halt,acc} indicates the end of the stream.

The basic logic is to use Task.start_link to start a long running process for each
item in the list that will send the result back to this process. Then, it
starts a stream that waits to receive a message from those long running
processes. Once it has received as many messages as there were items in the list
it ends the stream. Here the “accumulator” is just a count of the number of
messages that have been received so far.

I hope this description of using Stream.resource helps demystify creating
streams in your Elixir apps.

I talked earlier about creating and using a current_user in Phoenix,
but I didn’t explain
how to require a current_user for certain controller actions. In Rails I would
use a before_filter (or I guess now before_action). I wasn’t sure how to do
this in Phoenix, though.

This would run the require_user function before the new and create action,
just like you would do with a before_action in Rails.

Put code in web.ex to share it with all controllers

In Rails you can add methods to ApplicationController and, since all your other
controllers inherit from it, they will also have the same methods. As such, you
define general methods like require_user there so you can use them everywhere.

Elixir is not object oriented so you cannot use inheritance. Instead, there are
a series of macros inside the App.Web module that get run for various types of
modules. For example, the code inside the controller function gets put into
any module where you write use App.Web, :controller. As such, you can add code
to that function and all your controllers will “inherit” it.

What should the require_user function look like?

Currently, I have a module like this:

defmoduleApp.RequireUserdodefrequire_user(conn,_)doifApp.UserSession.current_user?(conn)doconnelseconn|>Phoenix.Controller.put_flash(:error,"You must be signed in to view that page.")|>Phoenix.Controller.redirect(to:"/")|>Plug.Conn.haltendendend

The general way it works is: if there is a current user, it just returns the
conn without changing anything. This means the controller action will happen
as normal. If there isn’t a user signed in, I add an error message to the flash,
redirect to the home page, and then halt the connection so no further code is
run. Depending on your app, you may just want to return a 403 status code and
not redirect or do something totally different.

Then, I add import App.RequireUser to the quote do block inside the
controller function in App.Web so all the controllers have access to it. I
could also just import App.RequireUser in each controller that needs it, but
all my controllers need it so it makes sense to do it in a single place for me.

I hope this further helps you dealing with the concept of current_user in your
Phoenix applications.

At Reverb.com we’ve adopted React as a way to start writing more javascript-centric code in our mostly more traditional server-rendered web app. I recently had to make a minor change to the very first React code I every wrote (all of 9 months ago) and realized that I’ve learned a lot about how to write clean React code since starting. I thought it would be helpful to share a little bit of what I now know with others.

1. Put state as high up as possible

If a component has state, nothing above it in the tree of components should care about that state. This means that most of your state should live in your top level component (or inside your flux store if you use some sort of flux library). There are cases where state might be kept in components not at the top level, but the state should be entirely internal. An example would be a component that manages a tabbed interface. It’s likely that nothing above that component in the component tree cares about which tab is active so state related to active tabs should be managed inside the tabs component.

Some clues that you might be on the wrong track are:

// Setting state from propsclassSomeComponentextendsReact.Component{constructor(props){super(props);this.state={aValue:props.aValue,};}// other code}

There are cases where this might be acceptable (for example you might set a
tabbed interface’s initial tab as props, but not care about which tab is
shown after that), but it is definitely a smell that something might be off.

// Setting state from props and then updating from new propsclassSomeComponentextendsReact.Component{constructor(props){super(props);this.state={aValue:props.aValue,};}willReceiveProps(nextProps){this.setState({aValue:nextProps.aValue});}// other code}

Don’t do this! If you have state that needs to get updated when props change
then that is clearly not state internal to this component. The state should live
further up in the component tree (and it apparently already does to some degree
since it keeps getting reset from props).

2. POJOs are your friend

The less logic you have in your React components, the easier they are to test and
reason about. I like to think of React components as a combination of a Rails
controller and view, so you want the components as skinny as possible
and to move the fat into models, which can be plain old javascript objects.
Generally, a plain object is easier to reuse and easier to set up (and thus
easier to test).

3. Smaller components are better

Similar to putting your logic into plain old javascript objects, making each of
your components smaller leads to easier reuse and easier understanding.
Ultimately, a whole complicated html page could be written as a single giant
component, but it will be impossible to reuse anything from it or understand any
of it after it has been written. I’ve learned that if a component has a lot of
logic or state, push most of the rendering into components that mostly just
render html.

I hope these ideas help you write cleaner, easier to use React code from the
start. Let me know if you have any other rules of thumb to avoid making a mess
in your React.

At Reverb we’ve been using Docker more and more. As part of this I’ve had to update my getting to work start-up script to start a Docker container, wait for the container to be up, and then execute some commands.

Unlike my vagrant-based start-up script that could execute vagrant up and wait for the command to complete, I’ll be running the docker-compose-up command and leaving that running in a tmux pane while working. This means that my previous tip of asking tmux to wait for the command to complete would not work because the command doesn’t actually ever complete.

Luckily, my coworker showed me a trick to check if the Docker container is up and available.

docker-compose up
until docker-compose exec psql -c "select 1" > /dev/null 2>&1; do sleep 2; done# other commands that assume the database is available

What this does is start the docker container, and then repeatedly trying to run select 1 against the postgres database inside the container. The command is meaningless (and the output is directed to /dev/null so I don’t see any error text from trying to run psql before the database is available) but if it successfully completes it means that the database is up and running inside the docker container and other commands that assume the container is running can be successful.

I hope this little trick is helpful to someone in writing some shell scripts that involve Docker.

While you still hear “fat models, skinny controllers” from time to time, people have generally caught on that not all code needs to be in a model or controller. Fat models don’t happen all at once, though. I find that you add a method here and a method there until you realize that one of your models is well over 200 lines and has too many methods and too little cohesion. Here’s how I approach this problem.

1. Look at the private methods

Assuming you factor shared code between public methods into private methods on your model, they are the best place to find related methods. Do you have two or more public methods using the same private method? Those methods could likely become the public interface for their own focused class. Also, if you have one public method using multiple private methods, it’s possible that method is so complicated it could be a whole class on its own.

Let’s say we have two public methods that share a private method like this:

classUser<ActiveRecord::Base# many methodsdefsubscription_active?most_recent_subscription.active?ifhas_subscription?enddefsubscription_expiration_datemost_recent_subscription.expiration_dateifhas_subscription?endprivatedefhas_subscription?subscriptions.any?enddefmost_recent_subscriptionsubscriptions.order(expiration_date: :desc)# this should be a named scope on Subscription, but it is here for example purposesend# many methodsend

The subscription_active? and subscription_expiration_date methods are prime candidates to get refactored into their own class.

classUser<ActiveRecord::Base# many methodsdelegate:subscription_active?,:subscription_expiration_date,to: :user_subscriptionsprivatedefuser_subscriptionsUserSubscriptions.new(self)end# many methodsend

At this point, make sure your tests all still pass. Then, move the tests for these methods from the User model to tests for the UserSubscriptions class. Make sure they still pass.

At this point, depending on the size of your code base and the number of callers of these two methods, you may end up stopping here. You may think you haven’t really improved things since you’ve just moved some code around and didn’t reduce the User model’s public footprint. You did, however, make an obvious place for future developers to add subscription-related code that isn’t inside the User model. This is an improvement because it discourages people from making the problems in the User model worse.

3. Change callers to use the new class

If at all possible, though, your goal should be to delete those delegations from the User model altogether. You do that by changing every instance of user.subscription_active? to UserSubscriptions.new(user).subscription_active?. You do the same with subscription_expiration_date and then you are well on your way to skinny controllers, skinny models.

I hope this helps you reduce the size of classes that get out of control!

Like many rails developers, I’ve been dabbling with elixir and the phoenix framework. While phoenix is clearly a lot like rails, it’s always an adjustment to learn a new framework and it is certainly an adjustment for me to get used to the functional programming paradigm coming from ruby.

One major philosophical difference between elixir and ruby is the attitude towards mocks and stubs. As elixir creator José Valim said:

I will fight against mocks, stubs and YAML in Elixir with all my… friendliness and energy to promote proper education on those topics.

This was a struggle for me since I was used to stubbing the current_user method in my rails controller tests to fake a user being logged in. I initially did Plug.Conn.put_session(:user_id, id) to store a user’s id in the session. This worked great, but was hard to use in tests because you can’t do put_session on a connection in controller tests without first making a request in the test. This slows things down for no real benefit.

I found this extremely helpful post that explains a ways to assign the current user to the connection directly in order to make testing easier. It doesn’t actually explain how to do this in the production code, though, since just assigning a current user to the connection doesn’t actually persist it to the session so the user won’t be logged in on subsequent requests. After some thinking I came up with a solution so I thought I’d write it down here to hopefully help myself in the future and other people learning elixir and/or phoenix.

What I’m doing here is storing the current user’s id in the session as well as assigning the user to the connection. When calling the current_user function if the connection does not already have a current_user assigned, I look up the user from the id stored in the session and assign that user to the session. This means that multiple calls to current_user won’t do a database lookup, but will just find it already assigned to the connection.

I added the following code to the test/support/conn_case.ex file inside the quote do block:

defpwith_current_user(conn,user)doconn|>assign(:current_user,user)end

This allows me to write tests assuming a user is logged in that start like:

user=create_user# assuming you have some function that creates a userconn|>with_current_user(user)|>get(some_path(conn,:get))

I hope this helps you write phoenix apps with less onerous test setup! If you have better solutions please let me know in the comments.

Database locking is a powerful feature of databases that can help prevent two people from modifying a resource in conflicting ways. Optimistic locking is when multiple users are allowed to read the same resource at the same time, but if more than one of them tries to modify that resource, the database prevents it. Pessimistic locking, which is the locking we are going to talk about today, actually prevents others from even reading it while it is locked.

After reading over that paragraph, I am confused so I assume you are too. I think it is much easier to learn with examples. Let’s take a look at some common use cases for pessimistic locking and how you can use it.

First, imagine you have a counter on a record that you need to increment. You might initially write code like this:

record=Record.find(1)record.counter+=1record.save!

This seems fine, but imagine if two users try to do this at the same time. They should each increase the value of counter by 1 for a total increase of 2, but since they both read the value and increment it at the same time, they will only collectively add 1 to the counter leading to inaccurate data. This is a classic race condition. Instead, you should ensure that only one user updates the record at a time with locking like so:

What with_lock does is a few things. First, it starts a database transaction. Second, it acquires a pessimistic database lock. Once the lock is acquired the record is reloaded in memory so the values on the record match those in the locked database row. The lock will prevent others from reading or writing to that row and anyone else trying to acquire a lock will have to wait for the lock to be released. This means that in our earlier example one of the two users would acquire the lock and increment the counter. The lock would be released at the end of the with_lock block and the other user would acquire the lock. Rails would refresh the data on the record so the counter reflects changes made by the first user and the record is now updated correctly.

Next, imagine you are making several changes and are already using a transaction. For example, maybe you are finding a user, updating her subscription, and updating her account balance to pay for that subscription:

Now, if the user tries to update two subscriptions at the same time they might use the power of the race condition to only pay once. If we are going to subtract from the account balance like this we clearly want to use pessimistic locking. Since we are already in a transaction, though, we don’t need to use with_lock and can instead do lock!:

This works very similarly to the with_lock method with two exceptions. First, it just acquires the lock at the time of calling and then releases it whenever the surrounding transaction completes rather than managing its own transaction internally. Second, it does absolutely nothing if not called inside of a transaction. To repeat, don’t use lock! outside of a transaction! Besides that, though, it will ensure the same type of data integrity that the with_lock method does.

Finally, maybe you have a table of discount codes that you give to each person who signs up for a newsletter. Each code starts out as available but when it is given to a user it becomes assigned. You might have code like this:

Our old friend the race condition returns with the possibility of two users getting the same discount code which could lead to havoc in your application. You need to ensure that each discount code is only assigned to one user. Pessimistic locking to the rescue:

This code is more complicated than our previous examples so it needs some explanation. Unlike the previous examples, in this case if we get a discount code that is locked by another user and then later released to us (because in this case once we acquire the lock it is no longer available even though we asked for the next_available_code) we need to get a new record. For this reason, we use recursion. It works like this: First, get the next available discount code. Second, acquire a lock on that discount code. Once the lock is acquired, check the state. If the discount code is still available, proceed as normal and assign it to that user. If it isn’t, then another person acquired a lock on this code first and another code should be acquired (hence the recursive method call).

I hope this helps you identify places where locking would help in your applications and help you understand how to use it.

According to a recent study, there are more than 100 hojillion websites on the internet. Due to the sheer number of options out there most people’s attitude when coming across a new site is:

Unfortunately not every site shows me what it got (I mean has) so I immediately navigate to one of my many other options on the web.

I know I normally write about programming topics, but when you are programming for a business on the internet you tend to develop some opinions about businesses on the internet. As a result, I’d like to share my thoughts with you.

1. Tell us what problem you solve

The bare minimum you can do is state the purpose of your site. Nowadays most sites manage to do this. One thing I’ve found, though, is a misunderstanding of what the purpose of the site is. Take mindmup.com for example. I came across this by following a link from the blog of one of the developers there. It states it is “Zero-friction free online mind mapping”. This is good in that it states the purpose of the site. I don’t know what mind mapping is, though, so I immediately left the website. Don’t tell me what your program does (i.e. makes mind maps), tell me what problems it solves. For all I know mind mapping would be perfect for some problem that I have, but unless my problem is that I can’t make mind maps (or the ways I have to make them cause too much friction) then mindmup.com doesn’t help me.

2. Show us how you solve that problem

“Show don’t tell” is a common phrase you hear. Since I have a child I now occasionally hear the phrase “show and tell” as well. Either way it is good advice. Take a look at gusto.com. They effectively explain that they make payroll and benefits easy. But then, they don’t tell you anything else until you either select how many employees are in your company or click “skip this question”. Frankly, I don’t want to tell you anything about myself until I’ve determined whether or not you can actually solve my problems. Gusto.com is basically standing behind a wall and saying, “Trust us, we can help you! Just give us your personal details first.” No thanks.

3. Make it memorable

I can’t just be all negativity, though, so let’s look at a good example, basecamp.com. It not only tells us the real problem it solves (organizing all the people involved with a project), it shows and explains how it solves that problem. Not only that, but it does it with a cute little logo and fun drawing. It helps you remember which website it was you looked at a few days later once you are actually ready to sign up for something.

It can be hard to take a look at your own site with the eyes of a person who doesn’t already know the domain and know about your site. Thinking about whether you are truly showing us what you got should help.