Why use a Javascript interface for Solr?

Search UIs are getting richer and more interesting by the day. Google page snapshots, animated disclosures of richer data on mouse over, and endless pagination are great examples. Often this type of functionality has a dark side. As each of these features is layered on and integrated with our HTML, it can begin to feel like death by a thousand paper cuts.

That’s where building a pure Javascript interface to search can start to help clean up and organize our code. At some point we manipulate the DOM so much we may as well have just created it in Javascript in the first place.

Once we have separated presentation from data we can begin to apply tools like design patterns (Visitor comes to mind) that make our code easier to read and modify.
** **

Pass the buck on rendering

*

Cloud meet Ocean.

On demand pages are expensive to render, and search is solidly in this category. Popular searches can be cached at the HTML level, but this is only only effective for very simple search behaviors and comes with its own set of problems. Cache expiry for content and what to do with Solr caching are two of these complications.

CSS is to HTML what Javascript is to Search. By giving the browser the ability to nicely render our data though Javascript we can refocus our servers on just serving the bits that change. Turning Solr results to JSON or XML is often much faster than rendering the same data as HTML. As a bonus, our presentation layer can be cached on the client giving us lower overhead on subsequent requests.

Even more server capacity can be reclaimed by letting the app server be a proxy for JSON results right out of Solr. This is an excellent way to get the most out of Solr’s own query caching.

Need more speed? Expiry headers and proxy caches like Varnish or Squid can still be used to further speed up the resulting JSON or XML requests.

Decoupling for Scaling

*
Better yet, what if we could scale search independently of content? Using techniques like Cross Origin Resource Sharing (CORS) or techniques like JSONP we can remove our app servers from the stack and go directly to the Solr servers.

Decoupling gives us much more freedom to make good scaling choices in server configuration. We can optimize search servers around serving small JSON documents and our app servers around supporting more traditional web content.

stacked vs decoupled servers

It’s also profoundly great for testing! Expectations for and of the search servers can be verified independent of the app server.

Solr doesn’t really have a provision for users and privileges which might have horrified you earlier when I implied you could make your Solr server public-facing. Fortunately this is easily managed.

Configure access to Solr endpoints on the web server. Some things to keep in mind:

Lock down any /update or /admin requests at this level

If you don’t plan on blocking /update and /admin here, move them to a different endpoint than the traditional Solr URI scheme and restrict access to an IP range or require authentication

Make sure to use invariants in the Solr server configuration if there are filter rules that must happen

Don’t put your security logic in the client side Javascript, reverse engineering and spoofing is possible even with compressed or obfuscated Javascript

Worried about someone requesting 1 million rows of data? Look into Apache’s mod_header or write a simple proxy app. This also works for preventing deep dives (start = 50000000)

SEO and Accessibility

You don’t have to worry about SEO on search pages.

Don’t believe me? Good. Who knows how Google and friends ultimately decide which content matches a query? If your SEO-practor says results pages matter then you will need a “Plan B.”

Use an AJAX request to set a cookie that lets your server know that the client supports Javascript and AJAX. From there you can decide to send them to an old-style search results page if needed. As long as the results are roughly the same Google has said this should be okay (and you don’t base this logic on browser identification).

And while you are at it, this page can be really helpful as a fallback to unconventional browsers like visitors with disabilities.

Getting Started

Check out the ajax-solr project if you’d like to start working with some of these methods.

More?

Let us know in the comments if you’d like to see more of our approach to Javascript and Solr.