Red Panthershttps://redpanthers.co
Experts in Ruby on Rails DevelopmentFri, 16 Nov 2018 17:51:03 +0000en-UShourly1https://wordpress.org/?v=4.9.8https://redpanthers.co/wp-content/uploads/2017/12/rp-logo.pngRed Panthershttps://redpanthers.co
3232Upgrading to Rails 5.1xhttps://redpanthers.co/upgrading-to-rails-5-1x/
https://redpanthers.co/upgrading-to-rails-5-1x/#respondWed, 21 Mar 2018 08:14:20 +0000http://redpanthers.co/?p=16171Rails 5.1rolled out with some major changes and features including Yarn and webpack support Dropped JQuery as a default dependency Built-in support for writing system tests using Capybara Encrypted secrets and many more, you can find the detailed release notes here. Recently we upgraded one of our production apps to 5.1x and since we have good test…

and many more, you can find the detailed release notes here. Recently we upgraded one of our production apps to 5.1x and since we have good test coverage, upgrading was mostly painless. We still had to make some minor changes down the road

1. No more before_filter

One of our controllers still had the old-fashioned before_filter and we encountered the following error while booting up

undefined method 'before_filter' before_filter has been deprecated in Rails 5.0 and removed in 5.1.

This is because before_filter and after_filter are deprecated from Rails 5.0.0 onwards. As you might have guessed, changing from before_filter to before_action is the solution here.

2. halt_callback_chains_on_return_false is deprecated

We saw the following deprecation warning in our logs after the update

ActiveSupport.halt_callback_chains_on_return_false= is deprecated and will be removed in Rails 5.2.

Though we had no return false in our callbacks, we still had to remove the line,

ActiveSupport.halt_callback_chains_on_return_false = false from config/initializers/new_framework_defaults.rb to get rid of the warning.

3. Passing a class to the class_name is deprecated

One of the deprecation warnings after the update was,

Passing a class to the 'class_name' is deprecated and will raise an ArgumentError in Rails 5.2.
It eagerloads more classes than necessary and potentially creates circular dependencies. Please pass the class name as a string:

This warning message was due to passing a class to class_name option when specifying associations – this is no longer recommended‘

PS: If you need any help in upgrading your rails version drop an email to us at info@redpanthers.co or upgrade@redpanthers.co. We will be more than happy to go through your system and give a free upgrade action plan and estimation .

]]>https://redpanthers.co/upgrading-to-rails-5-1x/feed/0What’s new in Ruby 2.5https://redpanthers.co/whats-new-ruby-2-5/
https://redpanthers.co/whats-new-ruby-2-5/#respondMon, 25 Dec 2017 08:08:56 +0000http://blog.redpanthers.co/?p=4219The Ruby language has increasingly stabilized over the years. The upcoming release of Ruby 2.5 is not going to let us down too. It introduces lots of new features and improvements over the previous version. The first preview was released on 10th October 2017 and the final build will be released on this 25th. This…

]]>The Ruby language has increasingly stabilized over the years. The upcoming release of Ruby 2.5 is not going to let us down too. It introduces lots of new features and improvements over the previous version. The first preview was released on 10th October 2017 and the final build will be released on this 25th. This blog dissects into this latest and exciting release and goes through some of the most important changes, we will be writing another article on performance improvement once the 2.5 if officially released.

Added Hash#transform_keys method

Ruby 2.4 added the transform_values method, 2.5 completes it by adding transform_keys thus make it a perfect pair.

Hash#transform_keys can change keys according to the return value of a block:

We know how many times we had to manipulate the keys of a hash. transform_keys is gonna be a game changer and going to be very compelling to be used in your legacy app. The destructive version is a just silver lining to this.

Array#prepend and Array#append

Ruby 2.5 brings home two new aliases which are so much better than the two of the most used operations in the language. Array#prepend and Array#append are more programmer friendly than the conventional Array#unshift and Array#push. Ruby IS THE language which focuses on the programmers’ happiness primarily after all.

Added yield_self method

This method yields the receiver to the given block and returns the output of the last statement in the block which is somewhat similar to the tap method.The only difference is the value that is returned.yield_self method returns the output of the block but tap method returns the receiver itself.

rescue/else/ensure are allowed inside do/end blocks without begin/end

We could omit the begin/end and not need the extra wrapper for rescue/else/ensure clauses in Ruby 2.5

[1].each do |n|
n / 0
rescue
# rescue
else
# else
ensure
# ensure
end

String#delete_prefix/delete_suffix

In Ruby 2.4 we used chomp method to remove the suffix ‘world’ from ‘HelloWorld’ and to remove prefix there is no corresponding method for chomp. The solution was to resort to a sub which is using the regular expression for such a simple task.

Ruby 2.5 added new methods to take care of such tasks. Now in order to delete prefix, we can use delete_prefix and to delete suffix we could use chomp. But the method names don’t seem good. So for symmetry delete_suffix was added.

Ruby 2.5 has removed top-level constant lookup

Consider the following code in Ruby 2.4.

class Book;
end
class Seller;
end
Book::Seller

This code works with a warning. The top-level constants are defined under Object class, and Ruby tries to look up the superclass of Book class, and eventually finds Seller under the Object class which is a superclass of Book class.

But in Ruby 2.5, Ruby won’t look up superclass. So the previous code fails with an error.

Dir.children and Dir.each_child

ls -a command will list all files including hidden files (files with names beginning with a dot). Dir.entries method present in Ruby 2.4 returns this output in an array.

Dir.entries("/home")
=> ["..", "user", "."]

Another method Dir.foreach iterates and yields each value from the output of ls -a command to the block.

Dir.foreach("/home") { |child| puts child }
..
user
.

The output includes the directives for the current directory and parent directory which are “.” and “..”.

When we want to have access only to the children files and directories, we do not need the [“.”, “..”] subarray. To overcome such issues, Ruby 2.5 introduced Dir.children. It returns the output of ls -a command without the directives for current and parent directories.

Dir.children("/home")
=> ["user"]

We can use Dir.each_child method to avoid yielding current and parent directory directives while iterating.

Dir.each_child("/home") { |child| puts child }
user

Imported features from ActiveSupport library

Over the past few years, Ruby has been merging the best features from the ActiveSupport library, into the core language. In Ruby 2.5 version, Hash#slice, Hash#slice!, Hash#except, Hash#except! are such methods continuing the trend, imported from ActiveSupport.

The ActiveSupport library comes bundled with the popular Ruby on Rails framework, but can also be used in isolation. It provides many extensions to Ruby’s core classes.

{a: 1, b: 2, c: 3}.slice(:a, :b)
#=> {:a=>1, :b=>2}

One of the notable feature in 2.5 release was bundler packed with ruby core, but it is posponed due to some issues, See the commit

References

]]>https://redpanthers.co/whats-new-ruby-2-5/feed/0Crystal tuples: the immutable data structure of crystalhttps://redpanthers.co/tuples-immutable-data-crystal/
https://redpanthers.co/tuples-immutable-data-crystal/#respondWed, 06 Dec 2017 13:47:11 +0000http://blog.redpanthers.co/?p=3985Tuple is a data structure which has immutable elements and is of a fixed size. Its similar to an array, but unlike arrays in crystal and ruby which allows adding more values over time a tuple is of fixed and cannot change. (Disclaimer: This article is meant for Ruby developer and explaining what a tuple…

]]>Tuple is a data structure which has immutable elements and is of a fixed size. Its similar to an array, but unlike arrays in crystal and ruby which allows adding more values over time a tuple is of fixed and cannot change. (Disclaimer: This article is meant for Ruby developer and explaining what a tuple is to a ruby developer).

In crystal we have two types of tuple

1) Tuple {1, "hello", :world}
2) NamedTuple {x: 1, y:2, z: 5}

They are immutable, which means if you try changing the value of an element in a tuple you will get an exception. Since crystal programs are compiled before execution you will get to see these errors while you compile the program itself.

example:

x = {1, 2, 3}
# to get the value use
x[0]

If you try assigning a value to it, like so

x[0] = 10

You will get an exception like bellow.

undefined method '[]=' for Tuple(Int32, Int32)

In crsytal, tuples are the preferred way to return a multiple results from a method.

For example inside the crystal core we have a method to get the minimum and maximum of an array.

(1..100).minmax

the result would be {1, 100}

Note: Since we just mentioned minmax, have a look at minmax_by method as well.It would let you apply a block of code over your range and then return the minimum and maximum based on the returned collection.

["1234", "12", "123"].minmax_by { |i| i.size }
# => {"12", "1234"}

Advantage of using tuple to return results instead of something like hash, is that we can be sure that our result cannot be altered accidentally. (since the data structure is immutable) ?

You can build a tuple from an array by using the .from method

Tuple(Int32, Int32).from([1, 2])

As a developer, the place where we use tuple the most in crystal are with splats(symbol: *). Passing arguments to method using splat and double splat operator is something we use widely in ruby keep our code small and readable. So if you wish to do the same in crystal you need to make a tuple not a hash or array.

If you use splat on an array directly like test(*[1,2]) it would return an error

argument to splat must be a tuple, not Array(Int32)

So to achieve the same effect as a splat with array in crystal we would need to do test(*{1,2})

Named Tuple

Named Tuple are everything as above, but with a name for each element. Named Tuple looks like {x: 1, y:2} it gives more meaning to our tuple. Like the above you can access the values but not change them.

Note: Crystal has a nifty feature called Union types (a variable can store data of multiple data types), so if it happen to pass such a variable to a named tuple/tuple, it will still check for the exact type that we want if the data is not in that variable then an exception would be raised

]]>https://redpanthers.co/tuples-immutable-data-crystal/feed/0Getting started with Faraday gemhttps://redpanthers.co/getting-started-faraday-gem/
https://redpanthers.co/getting-started-faraday-gem/#respondThu, 16 Nov 2017 17:54:51 +0000http://blog.redpanthers.co/?p=4033 Client libraries help in reducing the amount of code for the application developer who is using the API, whether a REST API or any other. By adding a set of code to the application, it provides the basic things an application needs to do in order to interact with the API. This is what a…

Client libraries help in reducing the amount of code for the application developer who is using the API, whether a REST API or any other. By adding a set of code to the application, it provides the basic things an application needs to do in order to interact with the API. This is what a client library does. Also, it may handle user authentication and authorization.

Among them, the favorite of mine is Faraday gem. Faraday has adapters for popular libraries like Net::HTTP. It is simple, flexible and supports multiple backends and embraces the concept of Rack middleware when processing the request/response cycle. Also, it’s possible to customize its behavior with middleware.

We will use a connection object to start with Faraday as it’s more flexible way than a simple getrequest.

Authentication

Basic and Token authentication are handled by Faraday::Request::BasicAuthentication and Faraday::Request::TokenAuthentication respectively. These can be added as middleware manually or through the helper methods.

conn.basic_auth('username', 'password')
conn.token_auth('token')

Proxies

To specify an HTTP proxy:

Faraday.new(:proxy => 'http://proxy.example.com:80')

Using a different HTTP Adapter

Faraday provides an interface between our code and adapter. Sometimes we may want to use features that are not covered in Faraday’s interface. In such cases, we can have access to features specific to any of the adapters supported by Faraday, by passing a block when specifying the adapter to customize it.

A new Faraday::Connectionobject is initialized. It has middlewares Faraday::Request::UrlEncoded in front of an adapter Faraday::Adapter::NetHttp. Like a Rack application, the adapter at the end of the builder chain is what actually executes the request.

Middlewares are grouped into request middlewares, response middlewares, and adapters.

Writing middleware

Middlewares are classes that respond to call. When middleware is executing, it’s passed as an env hash that has request and response information. Middleware wrap the request/response cycle.

The general interface for a middleware is:

class CustomizedMiddleware
def call(env)
# do something with the request
@app.call(env).on_complete do |env|
# do something with the response
end
end
end

All processing of the response should be done in the on-complete block. This enables middleware to work in parallel mode when many requests are occurring at the same time. After the on_complete block, env[:response] is filled in. Faraday::Response instance will be available onlyafteron_complete.

faraday-middleware is a collection of various Faraday middlewares for Faraday-based API wrappers.

For testing middleware, Faraday::Adapter::Test is an HTTP adapter middleware that lets you to fake responses.

References

]]>https://redpanthers.co/getting-started-faraday-gem/feed/0Getting start with React Nativehttps://redpanthers.co/getting-start-with-react-native/
https://redpanthers.co/getting-start-with-react-native/#respondMon, 23 Oct 2017 04:33:07 +0000http://blog.redpanthers.co/?p=3660React Native lets you build mobile apps using only JavaScript. A React Native app is a real mobile app. It uses the same design as React, letting you compose a rich mobile UI from declarative components. Installation Using Create React Native App we can start building a new React Native application easily. It allows you to start a project without installing or…

]]>React Native lets you build mobile apps using only JavaScript. A React Native app is a real mobile app. It uses the same design as React, letting you compose a rich mobile UI from declarative components.

Installation

Using Create React Native App we can start building a new React Native application easily. It allows you to start a project without installing or configuring any tools to build native code – no Xcode or Android Studio installation required.

Assuming that you have Node installed, you can use npm to install the create-react-native-app command-line utility:

import, from, class, extends, and the () => syntax in the example above are all ES2015 (also known as ES6) features. The unusual thing in this code example is.<Text>Hello world!</Text> This is JSX – a syntax for embedding XML within JavaScript. <Text> is a built-in component that just displays some text. This code is defining, App a new Component . Anything you see on the screen is some sort of component. A component can be pretty simple – the only thing that’s required is a render function which returns some JSX to render.

To modify your app, you can edit App.js file. The application should reload automatically once you save your changes.

Create React Native App

Create React Native App makes it really easy to run your React Native app on a physical device without setting up a development environment. If you want to run your app on the iOS Simulator or an Android Virtual Device, please refer to the instructions for building projects with native code to learn how to install Xcode and set up your Android development environment.

Once you’ve set these up, you can launch your app on an Android Virtual Device by running, npm run android or on the iOS Simulator by running npm run ios (macOS only).

Pros

JavaScript − You can use the existing JavaScript knowledge to build native mobile apps.

Community − The community around React and React Native is large, and you will be able to find any answer you need.

Code sharing − You can share most of your code on different platforms.

Cons

Native Components − If you want to create a native functionality which is not created yet, you will need to write some platform specific code.

React Native is focused solely on building a mobile UI. Compared with JavaScript frameworks like AngularJS or MeteorJS, React Native is UI-focused, making it more like a JavaScript library than a framework. The resulting UI is highly responsive and feels fluid thanks to asynchronous JavaScript interactions with the native environment. This means the app will have quicker load times than a typical hybrid app, and a smoother feel.

]]>https://redpanthers.co/getting-start-with-react-native/feed/0Observer Design Pattern in Rubyhttps://redpanthers.co/observer-design-pattern-ruby/
https://redpanthers.co/observer-design-pattern-ruby/#respondMon, 16 Oct 2017 10:24:44 +0000http://blog.redpanthers.co/?p=1264Observer design pattern (also known as Publish/Subscribe) is a software design pattern, used when we are building a system where the state of one object affects the state of other objects. It is a key part of model-view-controller architectural pattern. In a traditional MVC ( Model-View-Controller ) architecture, a model is a subject and a view is an observer.…

]]>Observer design pattern (also known as Publish/Subscribe) is a software design pattern, used when we are building a system where the state of one object affects the state of other objects. It is a key part of model-view-controller architectural pattern.

In a traditional MVC ( Model-View-Controller ) architecture, a model is a subject and a view is an observer. A view is notified when a model changes and responds accordingly. When the subject sends observers detailed information about what has changed, indiscriminately, this is known as the push model of the Observer pattern. When a subject sends only minimal information to its observers must ask for details explicitly, this is known as the pull model of the Observer pattern.

Ruby provides a simple mechanism to implement this design pattern using the Observable module. In this mechanism, the Notifier class uses the Observable module, which provides the methods for managing the associated observer objects.

Public Instance Methods

Instance methods are methods that are called on an instance of a class. We can use the below methods while using Observer instances.

add_observer(observer, func=:update)

Adds observer as an observer on this object, so that it will receive notifications.

changed(state=true)

Set the changed state of this object. Notifications will be sent only if the changed state is true.

changed?()

Returns true if this object’s state has been changed since the last notify_observers call.

count_observers()

Return the number of observers associated with this object.

delete_observer(observer)

Remove observer as an observer on this object so that it will no longer receive notifications.

delete_observers()

Remove all observers associated with this object.

notify_observers(*arg)

Notify observers of a change in state if this object’s changed state is true.

How it works

First, we have to create a basic structure of the Notifier class which will act as an Observer. The update() method is the callback that the Observable module will use when notifying changes to the observer, and the method name needs to be update().

Let’s take an example of an application which keeps track of the bike mileage and reminds us of when we need to take the vehicle in for a scheduled bike service.

class Notifier
def update(bike, miles)
puts "The bike has logged #{miles} miles, totaling #{bike.mileage} miles traveled."
puts "The bike needs to be taken in for a service!" if bike.service <= bike.mileage
end
end

By running the code we can see

bike = Bike.new(2300, 3000)
bike.log(100)
=> "The bike has logged 100 miles, totaling 2400 miles traveled."
bike.log(600)
=> "The bike has logged 300 miles, totaling 3000 miles traveled."
=> "The bike needs to be taken in for service!"

First, we create an instance of the Bike class with 2300 miles, and we set that it needs to be taken for service when it reaches 3000 miles.

]]>https://redpanthers.co/observer-design-pattern-ruby/feed/0Gulphttps://redpanthers.co/gulp/
https://redpanthers.co/gulp/#respondFri, 29 Sep 2017 09:36:51 +0000http://blog.redpanthers.co/?p=3697Gulp is a toolkit for automating painful or time-consuming tasks in your development workflow, so you can stop messing around and build something. You can compile sass files, uglify and compress js files and much more. Installation Make sure that you’ve installed Node and npm before attempting to install gulp. Install the gulp command [crayon-5bf33fbdbaf97718966242/] Make sure that…

]]>Gulp is a toolkit for automating painful or time-consuming tasks in your development workflow, so you can stop messing around and build something. You can compile sass files, uglify and compress js files and much more.

Installation

Make sure that you’ve installed Node and npm before attempting to install gulp.

Install the gulp command

npm install --global gulp-cli

Make sure that you have your package.json created by manually creating it or typing npm init.

Run this command in your project directory:

npm install --save-dev gulp

Create a gulpfile

In your project directory, create a file named gulpfile.js in your project root with these contents:

The first step to using Gulp is to require it in the gulpfile. Then you can begin to write a gulp task with this gulp variable. task-name refers to the name of the task, which would be used whenever you want to run a task in Gulp. You can also run the same task in the command line by writing gulp task-name.

Run the gulp command in your project directory:

gulp

To run multiple tasks, you can use gulp <task> <othertask>

Gulp tasks are usually a bit more complex than this. It usually contains two additional Gulp methods, plus a variety of Gulp plugins.

Now, anytime you make changes and save any file with a .scss extension, it will trigger this [‘watch’]task which executes [‘sass’].

Gulp is a task runner that uses Node.js as a platform. Gulp purely uses the JavaScript code and helps to run front-end tasks and large-scale web applications. It builds system automated tasks like CSS and HTML minification, concatenating library files, and compiling the SASS files. These tasks can be run using Shell or Bash scripts on the command line.

Reference

]]>https://redpanthers.co/gulp/feed/0Background Workers using Crontabhttps://redpanthers.co/background-workers-using-crontab/
https://redpanthers.co/background-workers-using-crontab/#commentsWed, 27 Sep 2017 06:17:21 +0000http://blog.redpanthers.co/?p=3910Scheduling background jobs is a common task in rails application development. Eventually what we want is a cron job which runs the schedule jobs. cron is the system process which will automatically perform tasks for you according to a set schedule. The schedule is called the crontab, which is also the name of the program…

]]>Scheduling background jobs is a common task in rails application development. Eventually what we want is a cron job which runs the schedule jobs.

cron is the system process which will automatically perform tasks for you according to a set schedule. The schedule is called the crontab, which is also the name of the program used to edit that schedule.

For example, let’s say you have a rake task which you want to run every hour.

namespace :send_update_mail do
desc "send_product_update_mails"
task :send_mail => :environment do
UserMailer.notify_product_updates
end
end

To edit the crontab, use this command:

crontab -e

Now let’s add our job to the crontab. Each job you add should take up a single line.

The format is very simple: six pieces of information, each separated by a space; the first five pieces of information tell cron when to run the job, and the last piece of information tells cron what the job is.

m h dom mon dow command

m, representing the minute of the hour, h, representing the hour of the day, dom, representing the day of the month, mon, representing the month of the year, dow, representing the day of the week and command, which is the command to be run. For example in our case

0 * * * * /home/myname/myapp/lib/tasks/send_mail.rb

The asterisks (“*“) will tell cron that for that unit of time, the job should run ‘every’. Save and close the file.

And that’s it.

But sometimes fiddling with crontab on the server can be very hectic and it would be much better if we can configure cron job in our rails application so we can keep the configuration in our source control.

We have a gem called whenever that allows us to set up cron jobs from within our Rails apps using Ruby code. Let’s see how you can schedule our background jobs in Rails using Whenever to set up your schedule.

Add in your Gemfile.

gem 'whenever', :require => false

Run bundle install to install the gem.

Run the wheneverize command in your app’s root folder to set up an initial configuration.

wheneverize .

The wheneverize command will create an initial config/schedule.rb file.

A whenever plugin for mina

require 'mina/whenever'
desc "Deploys the current version to the server."
task :deploy do
deploy do
.........
on :launch do
invoke :'sidekiq:restart'
.....
end
end
end

Drawback

This rake task-based approach can have a potential drawback of extra memory consumption. When crontab runs rake or rails runner, it is booting up a full instance of our rails application to access relevant models and associations. It is memory expensive especially when your code base is huge and using a considerable number of gems. In a nutshell, the entire app and dependencies are loaded for every task that runs.

Imagine if you have scheduled n number of background workers to start at a given point of time in our main application server. This could end up being too resource expensive that our application might become unusable for the user.

How to fix?

1) Start another server to schedule jobs and another server to consume the jobs.

2) Create another Ruby project that would add the messages to our Redis queue to schedule the tasks.

3) Offload that to another language like Crystal, which is more efficient and schedule jobs independent of our Rails app.

4) Use a sidekiq or other background job to schedule which takes less RAM

1st would be the easiest solution as it would keep our codebase same. (Heroku works like this)
2nd and 3rd are recommended if you are looking to keep it on a single server and keep the cost down until the business scale.

References

]]>https://redpanthers.co/background-workers-using-crontab/feed/4Cheat sheet for managing files from Rubyhttps://redpanthers.co/cheat-sheet-for-managing-files-from-ruby/
https://redpanthers.co/cheat-sheet-for-managing-files-from-ruby/#commentsFri, 22 Sep 2017 17:34:15 +0000http://blog.redpanthers.co/?p=1596In this Cheat sheet, you will learn managing files from Ruby. Files are used for storing the Data for a Long time Period. And the files can contain any type of information means they can Store the text, any Images or Pictures or any data in any Format. It is associated with class IO File…

Files are used for storing the Data for a Long time Period. And the files can contain any type of information means they can Store the text, any Images or Pictures or any data in any Format. It is associated with class IO File includes.

Creating a New File

New files are created in Ruby using the new method of the File class. The new method accepts two arguments, the first being the name of the file to be created and the second being the mode in which the file is to open. Like,

file = File.new(“filename”, “mode”)

Eg:

file = File.new("file.txt", "w")

Supported modes are:

r

Read only access.

r+

Read and write access.

w

Write only access.

w+

Read and write access.

a

Write only access.

a+

Read and write access.

Opening Existing Files

You can open the existing files using the open method.

Eg:

file = File.open("temp.txt")

If the file is already opened, we can close it by using the close method.

Eg:

file.close

Reading and Writing Files

Once we’ve opened an existing file or created a new file we need to be able to read from and write to that file. We can read/write using different methods.

]]>https://redpanthers.co/cheat-sheet-for-managing-files-from-ruby/feed/1Get into Sentiment Analysis with Rubyhttps://redpanthers.co/get-into-sentiment-analysis-with-ruby/
https://redpanthers.co/get-into-sentiment-analysis-with-ruby/#respondThu, 21 Sep 2017 13:46:35 +0000http://blog.redpanthers.co/?p=3497 Sometimes we fail to understand other’s emotion. So how it will be when machines try to understand ours? When writing programs we care about the syntax and structures but these concerns are not there in communication between people. To process our language machines have to understand not only what we say, but what we…

Sometimes we fail to understand other’s emotion. So how it will be when machines try to understand ours? When writing programs we care about the syntax and structures but these concerns are not there in communication between people. To process our language machines have to understand not only what we say, but what we mean. Natural language processing is a fascinating subject to explore. But what makes it complicated?

Human communication isn’t just a group of words. It’s a mix of sentiments which needed to be analyzed to understand what we really mean.

Why should I care

The contemporary business world is a place where huge success and failure sit side by side. In traditional market research, business spends a huge amount to analyze customer’s opinions through continuous surveys and consultants. But nowadays social media empowers business a lot. Most the existing and potential customers are generating a treasure trove of data through Twitter, Facebook, LinkedIn and so on. Sentiment analysis is a powerful tool for mining the gold beneath the social media landslide.

The goal of sentiment analysis is to identify the opinions expressed in a text.

It seems easy, right?

I’m happy to watch a new movie

I hate war

Since happy is a positive word and hate is negative, we know the sentiments in each sentence. This is the most simple situation.

But what if a text contains two opinion words and one sentiment?

If we assign +1 to positive word and -1 to negative word,

I’m happy and excited to be going to watch a new movie

The above sentence is positive +2

I hate war and the violence it makes

The text contains two negative words so it is negative -2

Now, let’s consider a text with mixed polarity.

That’s, two opinion words and two sentiments

What we learned is adding +1 and -1 equals 0 but here it doesn’t mean that the text is necessarily neutral.

Ice cream shops are doing great even when the weather is bad

The statement is positive about ice cream shops but negative about weather condition. In general, we will say that this is a positive statement about ice cream shops. We wouldn’t say this is neutral.

When SA turns hard

Sometimes it’s not possible to identify the opinion by just analyzing the polarity of words. Language usage and sarcasm are some of the reasons why sentiment analysis turns hard. It’s tough to analyze mixed sentiments in a text.

Sometimes it’s difficult to classify sarcastic statements as positive or negative. As a human being, we can understand what sarcasm is and how it actually makes sense. If you put this into a neural network or any machine learning framework that come up with a simple classifier to just understand the sentiment, this would fail miserably.

Another point to be considered is local dialects. If you train any neural network on data about local dialects, it would invariably not understand what is trying to say. Because some of the words in local dialects may not have any sense and its tough to train anything and everything. So if you have a pre-trained model doing some sort of test on these data and then, it would completely fail. This is one of the reasons why it’s important to understand the local culture and some companies are setting up local data centers, where local sentiment is captured.

Natural language has a lot of ambiguity. From the above examples, it’s clear that words make sense contextually in natural language which humans can comprehend and distinguish easily, but machines can’t. This makes Natural Language Processing one of the most difficult and interesting tasks in AI.

Using Natural Language Processing

Spell check and grammar check

Predictive text

Auto summarization

Machine translation

Sentiment analysis

Some common approaches to sentiment analysis

Various methods in Machine learning and Natural Language Processing for sentiment analysis. Some of the most effective approaches we have today rely on the human-in-the-loop-approach: learning from the user feedback. The combination of machine-driven classification enhanced by human-in-the-loop approach increases acceptable accuracy than pure Machine Learning based systems.

Tools & Libraries

Python’s scientific calculation libraries such as SciPy, NumPy have strong support from the academic world. It’s a very well established library that was chosen for its expressiveness, ease-of-use.

We too have tools……

Much of the data that machine learning algorithms need for NLP tasks such as sentiment analysis, spam filtering all come from the web. Ruby has a web framework that is quite popular and generates massive amounts of data. While it doesn’t have the same vast academic network that Python or R has, it does have tools and has the added benefit of being easy to learn and comprehend.

Sentimental gem

Sentimental gem was introduced for simple sentiment analysis with Ruby. It implements a lexicon-based approach to extract sentiments, where the overall contextual sentiment orientation is the sum of sentiment orientation of each word(tokens). The overall sentiment of a sentence is output as positive, negative or neutral. It uses a dictionary consisting of pre-tagged lexicons. The input text is converted to tokens by the Tokenizer and is then matched for the pre-tagged lexicons in the dictionary.

To classify sentiments we can set a threshold value. Values greater than the threshold is considered as positive and less than that is considered negative. The default threshold is 0.0. If a sentence has a score of 0, it is deemed “neutral”.

The overall score is determined by the sum of the scores of each opinion words. In its lexical dictionary, good is assigned a score 0.6394, bad is assigned -0.5588, and the token weather is assigned a score of -0.5. Hence the overall sentiment scores -0.4194.

The gem was found to work well for simple sentences, but failed to give accurate results for sentences with mixed polarity.

Sentimentalizer gem

Implements sentiment analysis in Ruby with machine learning. It’s basically training a model to use in the application. Machine learning based analysis gains more interest of researchers due to its adaptability and accuracy. It overcomes the limitation of the lexical approach of performance degradation and works well even when the dictionary size grows rapidly.

But this method faces challenges in designing classifier, availability of training data, the correct interpretation of a new phrase which is not in the training dataset.

Classifying with Bayesian and SVM classifiers

Basically, sentiment analysis is the classification of text. Ankusa, Eluka, Classifier, and Hoatzin are some Bayesian and SVM classifiers that can be used for sentiment analysis. Among them, Hoatzin, Classifier, and Eluka use LibSVM, a library for Support Vector Machine. Simple models work best for all. The gem Ankusa provides Naive Bayes classifier which provides more accuracy than Baseline but less than gem Eluka which implements SVM classifier.

When we need more…

JRuby

Ruby is a very expressive language with excellent string processing capabilities. Also, there are excellent Java libraries for NLP and JVM is a high-performance platform with true multi-threading capabilities. JRuby allows us to leverage well-established, mature Java libraries from within your Ruby code.

Sentiment Analysis using Tensorflow Ruby API

TensorFlow is an extraordinary open source software library for numerical computation using data flow graphs developed by researchers working on the Google Brain Team within Google’s Machine Intelligence research organization for conducting machine learning and deep neural networks research. Even though Tensorflow seems to be an overkill for simpler tasks, certainly it would be an alternate and more efficient way to analyze tweets if you have rich and high volume data. It helps to create your own sentiment classifiers to understand the large amounts of natural language in the world.

In this article, we discussed the various approaches towards sentiment analysis which is a part of Natural Language Processing. We have seen that sentiment extraction and analysis can be done using supervised or unsupervised learning, sentiment lexicon-based approach or a mix of these and any of these methods can be implemented in Ruby.