Rangarajan Krishnamoorthy on Programming and Other Topicshttps://www.rangakrish.com
Perspectives on Software, Music, Photography, Homeopathy, etc.Wed, 20 Mar 2019 13:57:29 +0000enhourly1https://wordpress.org/?v=5.0.3145085559Controlling Reaktor from Opusmodus Using OSChttps://www.rangakrish.com/index.php/2019/03/20/controlling-reaktor-from-opusmodus-using-osc/
https://www.rangakrish.com/index.php/2019/03/20/controlling-reaktor-from-opusmodus-using-osc/#respondWed, 20 Mar 2019 13:40:28 +0000https://www.rangakrish.com/?p=1501I am a great fan of Opusmodus software. I have written many articles about it in the past. Owing to my several other commitments, I am not able to spend much time on it these days, but I keep checking the features in new releases regularly. The software has evolved substantially since its first release a few years ago.

One of the nice feature additions in the latest 1.3 release is support for Open Sound Control (OSC). This is something I have been waiting for a long time, so I decided to check it out immediately.

In an earlier article, I showed how it is possible to configure Reaktor as a OSC receiver. For that example, I used Chuck programming language to send OSC commands to Reaktor. Today, I want to reconstruct the same Reaktor experiment, but using Opusmodus, instead of Chuck, to send OSC messages.

Make sure you are running the latest release of Opusmodus. As of this writing, the latest version is 1.3.24692.

First, we launch Reaktor 6, and create a new Sine oscillator with Pitch and Amplitude controllers. The output can be connected to the default output channels. See the following figure:

A Simple Sine Oscillator

Next, we should configure the Pitch and Amplitude controllers to receive OSC messages. Whatever name we use here must be the same as the one we use in Opusmodus. We choose the name “pitch” for the Pitch controller and “ampl” for the Amplitude controller.

Setting Up OSC Address

Now that Reaktor has been setup, we can run Opusmodus and send OSC messages. Here is a very short code fragment that illustrates the idea:

Sending OSC Messages

The actual function that transmits the messages is “osc-thread”. The first argument to the function is the destination address: IP address and Port number. The second argument is the OSC address of the receiving controller (make sure the name matches that in Reaktor). The last argument is the actual data to be transmitted to the recipient. It is a series of <value> and <time> pairs.

The “osc-thread” function returns a Thread object. We save this in a variable so that we can stop the thread if needed. The created thread runs independent of other threads. In our case, we have created two threads to take care of Pitch and Amplitude controls in Reaktor.

In this example, I have hardcoded the data values to be sent to Reaktor. In practice, this data is likely to be generated algorithmically elsewhere. Opusmodus has a rich set of functions to do this. Here is another code fragment that uses “gen-osc-data” to get the data in the required format before sending to the target:

Using “gen-osc-data” Function

You can see the data being received by Reaktor:

Monitoring OSC Messages

That is it. Quite straightforward, isn’t it? What I have demonstrated in this example is an extremely simplified situation. It is possible to setup more complex ensembles in Reaktor and control them from Opusmodus using OSC. But the idea is the same. If time permits, I will discuss more interesting examples in the future.

I would like to thank Janusz Podrazik, the creator of Opusmodus, for developing such a nice environment for music composition, and more importantly for his great support!

Author: Anthony Williams

Publisher: Manning Publications Co.

Year: 2019 (Second Edition)

The first edition of this book came out in the year 2012 and covered the concurrency features of C++ 11. Much has happened since then. The present C++17 is significantly richer in terms of concurrency support, especially in the area of parallel algorithms. The release of the second edition of the book recently is a nice addition to the growing number of good books on C++ 17.

The book contains 11 chapters. The major difference between this edition and the earlier one is that Chapter 10 on “Parallel Algorithms” is completely new in this second edition.

The first two chapters touch upon the basics of threading, including creating threads, waiting for threads to complete, and transferring ownership of threads. There is also a brief discussion on how to determine the number of threads supported by the hardware so that one can dynamically decide the optimal number of threads to spawn.

Chapter 3 talks about the problems of sharing data across multiple threads and the typical use of mutexes to do this correctly. Some interesting, but less common, cases can be handled through the use of std::call_once() function. This is also described in this section.

Synchronizing concurrent operations is the focus of the fourth chapter. Condition variables, Futures and Promises are covered here. One of the interesting sections in this chapter is about the continuation-style concurrency as specified in the Concurrency TS. This functionality, although not yet in the standard, is available through the std::experimental namespace.

Chapter 5 covers the important topic of C++memory modelfrom the perspective of threading. I would even go to the extent of saying that this should have been the first chapter of the book, because everything else depends on this foundation.

Chapters 6 and 7 discuss how one can design concurrent data structures that are both thread-safe and allow genuine concurrency. Although the general advice is to use standard data structures and algorithms from the library, often times, we have to implement our own data structures that are domain specific. These two chapters explain how to achieve this.

In Chapter 8, the author gives broad guidelines on how to design concurrent code. Important concerns about performance, scalability and exception safety are discussed here. These are practical issues that a developer needs to be aware of.

Chapter 9 touches upon the finer aspects of thread management. The two key concepts covered in this chapter are thread pooling and thread interruption. Both are important and are widely used in real-world applications.

Chapter 10 is about the support for parallel algorithms in STL. This functionality is one of the major additions in C++17.Several STL algorithms, including copy, find, sort, and transform can now execute in parallel. This chapter gives examples using the different execution policies std::execution::seq, std::execution::par, std::execution::par_unseq.

Testing and debugging multithreaded code is quite a challenge. Chapter 11 covers this topic and has some useful tips.

There are 4 appendices at the end of the book. Appendix A is titled “Brief reference for some C++11 language features”. This seems a bit odd in a book that discusses concurrency as supported in C++17. The author could have added some features of C++14 and C++17 to this chapter (there is just one additional topic “Class Template Argument Deduction” compared to the previous edition).

Appendix B provides a single-page comparison of the concurrency features of “Java”, “Posix C”, “Boost Threads”, and “C++11”. This part has been retained as it is from the First edition. I would have liked it if it had been updated to the current standard, and some additional languages included (for example, concurrency in C#, go, etc.).

Appendix C contains the complete source code for the message-passing framework and ATM example discussed earlier in the book. I am surprised that even this code is identical to the earlier version. The author could have used this larger example to showcase some of the newer concurrency features covered in the book.

The last appendix, spanning 150 pages, is a reference to the C++thread library. Although it can be handy when you are reading the book, I personally prefer the online reference, for example, this.

Overall, this is a very useful and readable book, one that every C++ developer must have. However, having read the first edition, I feel the author could have added some more useful material in this revised edition; it feels like he was in a hurry to release the book because it was already overdue. Of course, if you do not have the first edition, now is the best time to get this revised edition.

Have a nice weekend!

]]>https://www.rangakrish.com/index.php/2019/03/10/book-review-c-concurrency-in-action/feed/01496Text Summarization APIshttps://www.rangakrish.com/index.php/2019/03/03/text-summarization-apis/
https://www.rangakrish.com/index.php/2019/03/03/text-summarization-apis/#respondSun, 03 Mar 2019 05:32:21 +0000https://www.rangakrish.com/?p=1490I talked about detecting Emotion from text in the last two articles. Another popular text analysis service is Text Summarization.

There are two approaches for summarization:

Extractive summarization

Abstractive summarization

In the first approach, “Extractive Summarization”, the system extracts key sentences from the given text and puts them together to form a summary. There are no new words or phrases in the summary. This is the widely used approach today. An algorithm called “TextRank” is the inspiration for many such implementations. You can read more about this approach here.

The second approach, “Abstractive Summarization” attempts to “understand” what is being discussed in the given text, in order to generate a summary that usually includes new words and phrases. Remember, we humans are good at summarizing given text “in our own words”. This is a challenging research problem, and a lot of focus is currently on this topic.

Today, I am going to take a sample text and run it through three different Summarization APIs and look at the output generated by them. The three API services I am going to use are:

I have written an article earlier describing MeaningCloud’s Parsing API. This Madrid-based company offers a variety of text analysis APIs.

The input for summarization is an article based on Florence Nightingale and is taken from this page.I copy-pasted the text onto NotePad (Windows app) and removed a couple of non-ascii characters so that the final text has only ascii characters (this is not a requirement for generating summary, though).

MeaningCloud.com

Working with MeaningCloud was easy. I logged into my account and pasted the input text in the “Test Console” corresponding to Text Summarization API. The nice thing is that there is an option to specify the number of sentences in the generated summary, and I chose the default (5 sentences). Here is the generated summary:

Nightingale was helped to understand statistics by her country’s leading expert on public statistics,. Cholera is caused by a bacterium spread by people drinking water contaminated by sewage. Florence Nightingale was born into an upper class British family in 1820 in Florence, Tuscany, Italy. In 1907, Florence Nightingale became the first woman to be awarded the Order of Merit by King Edward VII. In London and a Florence Nightingale museum, also in London.

Just to make it interesting, I ran the summarizer again, with the option to emit 10 sentences this time. Here is the result:

Florence Nightingale, OM (12 May 1820 – 13 August 1910), was an English nurse. Florence Nightingale was a wonderful woman. Nightingale was helped to understand statistics by her country’s leading expert on public statistics,. Cholera is caused by a bacterium spread by people drinking water contaminated by sewage. Florence Nightingale was born into an upper class British family in 1820 in Florence, Tuscany, Italy. In 1854 when the Crimean War began, Florence was working in Harley Street in London. Hospital in Istanbul where the injured soldiers were sent, Florence realized that soldiers died more often. In 1907, Florence Nightingale became the first woman to be awarded the Order of Merit by King Edward VII. Nightingale died in 1910 in London. In London and a Florence Nightingale museum, also in London.

Aylien.com

Just as I did with MeaningCloud, I logged into Aylien’s account and submitted the article. In this case, I had to submit the URL of the article. I chose the default summary size of 5 sentences and here is the output :

Florence Nightingale, OM (12 May 1820 – 13 August 1910), was an English nurse.

In 1854 when the Crimean War began, Florence was working in Harley Street in London.

At the hospital in Istanbul where the injured soldiers were sent, Florence realized that soldiers died more often from diseases like cholera than from their injuries in war.

She used her knowledge of maths and statistics to show the British government that providing better conditions for sick and injured soldiers would help them win the war.

There are many statues of her in Britain, including one in Waterloo Place in London and a Florence Nightingale museum, also in London.

When I chose 10 sentences as the summary size, this is what I got:

Florence Nightingale, OM (12 May 1820 – 13 August 1910), was an English nurse.

She became a leader of the team of nurses who helped wounded soldiers during the Crimean War.

Florence Nightingale was a wonderful woman who fought the odds of not living a life expected by her family.

Florence Nightingale was born into an upper class British family in 1820 in Florence, Tuscany, Italy.

In 1854 when the Crimean War began, Florence was working in Harley Street in London.

At the hospital in Istanbul where the injured soldiers were sent, Florence realized that soldiers died more often from diseases like cholera than from their injuries in war.

She used her knowledge of maths and statistics to show the British government that providing better conditions for sick and injured soldiers would help them win the war.

Nightingale believed religion helped provide people with the fortitude for arduous good work.

In 1907, Florence Nightingale became the first woman to be awarded the Order of Merit by King Edward VII.

There are many statues of her in Britain, including one in Waterloo Place in London and a Florence Nightingale museum, also in London.

Deepai.org

In the case of Deep AI, there is no option to control the number of sentences in the generated summary. The documentation says that the summary is about 20 percent of the original text. I logged into my account and pasted the input text in the test console. Here is the generated summary:

Florence Nightingale, OM (12 May 1820 – 13 August 1910), was an English nurse.

helped wounded soldiers during the Crimean War.

Florence Nightingale was a wonderful woman

Nightingale was helped to understand statistics by her country’s leading expert on public statistics,

Unfortunately, both she and Farr believed the disease was caused by foul air: this was called the miasma theory.

Florence Nightingale was born into an upper class British family in 1820 in Florence, Tuscany, Italy.

In 1854 when the Crimean War began, Florence was working in Harley Street in London.

hospital in Istanbul where the injured soldiers were sent, Florence realized that soldiers died more often

show the British government that providing better conditions for sick and injured soldiers would help them

There is a syndrome named after her called “Florence Nightingale Syndrome”.

Nightingale believed religion helped provide people with

In 1907, Florence Nightingale became the first woman to be awarded the Order of Merit by King Edward VII.

Nightingale died in 1910 in London.

in London and a Florence Nightingale museum, also in London.

The summary has 14 sentences.

Observations

Let us discard the 5-sentence summaries from MeaningCloud and Aylien and consider only the 10-sentence versions. That way, the output from all three API services are of “similar” size.

You can detect substantial similarity in the summaries generated by MeaningCloud and Deep AI. In fact, except for the sentence “Cholera is caused by a bacterium spread by people drinking water contaminated by sewage.”, every sentence in the summary of MeaningCloud is also in the summary of Deep AI. Of course, Deep AI has a few extra sentences (total size is 14) as I pointed out earlier.

What is interesting is that some of the sentences in both the summaries are truncated versions of the original sentences. For example, the actual sentence

“Nightingale was helped to understand statistics by her country’s leading expert on public statistics,William Farr.”

has been trucated to:

“Nightingale was helped to understand statistics by her country’s leading expert on public statistics,.”

As another example, the original sentence

“She became a leader of the team of nurses who helped wounded soldiers during the Crimean War.”

has become:

“helped wounded soldiers during the Crimean War.”

It is not clear why this happens.

In terms of the overall output, I prefer the summary produced by Aylien. It appears a bit more coherent than the other two.

What if we asked two people to generate “extractive summarization” manually? Will their output match that of Aylien? My guess is that there is likely to be variation even among human generated summaries, because identifying “important” sentences in a given piece of text is somewhat subjective. Secondly, even if two people choose the same set of sentences, they might rearrange them in slightly different order.

In the meantime, while we are eagerly waiting for good quality “abstractive summarization” implementations, we have to make do with “extractive summarization”.

Have a nice weekend!

]]>https://www.rangakrish.com/index.php/2019/03/03/text-summarization-apis/feed/01490Emotion Detection using ParallelDots APIhttps://www.rangakrish.com/index.php/2019/02/24/emotion-detection-using-paralleldots-api/
https://www.rangakrish.com/index.php/2019/02/24/emotion-detection-using-paralleldots-api/#respondSun, 24 Feb 2019 05:14:03 +0000https://www.rangakrish.com/?p=1475Last week, I showed how we can use IBM Natural Language UnderstandingAPI to identify emotions from given text. Today, I would like to run through the same examples, but using ParallelDots APIservice.

There are wrappers in Java, Python, Ruby, C#, and PHP for accessing the REST service. However, I chose to write my own implementation in Lisp (you can get the program here). The core functions are shown below:

The Code

I think the code is self explanatory, so I won’t go into the details.

Before we try our sample sentences, we have to create a “paralleldots” object first:

CL-USER 1 > (setf pd (paralleldots +PARALLELDOTS-API-KEY+))

#S(PARALLELDOTS :API-KEY “<Key>” :RESULT NIL)

Substitute your API key instead of “+PARALLELDOTS-API-KEY+”.

Here is our first sentence:

“It was scary to drive alone on the highway.”

We submit this for emotion detection by invoking the “emotion” method:

The result contains the dominant emotion followed by the individual emotions. For some reason, even though the company’s web site lists seven emotions, the returned result contains only six emotions (the emotion “sarcasm” is not included).

As the above result shows, “Fear” (0.57) is the dominant emotion. I don’t understand why “Sad” (0.35) comes fairly close in the second place. If you recall IBM’s analysis for this example,“Fear” was 0.90 and “Sadness” was just 0.11. That appears reasonable.

The actual result returned by the API contains status code as well; for completeness, I have included a method to dump the actual result of the API call.

CL-USER 3 > (pprint (result pd))

((:CODE . 200)

(:EMOTION

(:PROBABILITIES

(:*FEAR . 0.57426417)

(:*SAD . 0.35364214)

(:*HAPPY . 0.0052627796)

(:*ANGRY . 0.044279967)

(:*EXCITED . 0.018575214)

(:*BORED . 0.003975709))

(:EMOTION . “Fear”)))

As you can see, the earlier shown result extracts just the emotion details from this more detailed result.

Let us move on to the next sentence:

“I can’t wait to see the President in person!”

Here is what the system says:

CL-USER 4 > (emotion pd “I can’t wait to see the President in person!”)

“Excited” is the main emotion. Again, “Fear” comes close and I don’t understand why. Even in IBM’s case, I had pointed out that the “Joy” factor was just 0.32, whereas it should have been higher in my opinion.

Now, the third sentence:

“The talk was dull and uninteresting. The audience was literally yawning throughout the program.”

The analysis is:

CL-USER 5 > (emotion pd “The talk was dull and uninteresting. The audience was literally yawning throughout the program.”)

“Angry” (0.58) predominates, with “Sad” coming at a distant second (0.21). This seems OK to me. In IBM’s case, we had “Disgust” (0.62) and “Anger” (0.38).

Overall, ParallelDots does a decent job of identifying emotions from the given text. Although my focus in this article has been on “Emotions“, they have APIs to handle “Sentiment“, “Abuse“, “Intent“, etc. The company has a “Free” plan that includes 1000 hits per day, which is quite sufficient for getting started and for exploring their offering. Do give it a try.

]]>https://www.rangakrish.com/index.php/2019/02/24/emotion-detection-using-paralleldots-api/feed/01475Identifying Emotions from Texthttps://www.rangakrish.com/index.php/2019/02/17/identifying-emotions-from-text/
https://www.rangakrish.com/index.php/2019/02/17/identifying-emotions-from-text/#respondSun, 17 Feb 2019 03:30:42 +0000https://www.rangakrish.com/?p=1460Identifying the predominant sentiment in unstructured text is used widely these days. There are several REST API services that allow you to submit a piece of text and get back the corresponding sentiment analysis. Meaningcloud, Aylien, Google’s Cloud Natural Language API, andIBM Natural Language UnderstandingService are just a few.

Emotion detection, especially from facial expressions and speech, is also catching up. Google and Microsoft have APIs to do this. However, for my article today, the focus is on detecting emotions from “unstructured text“, and I am going to look at IBM’s API as part of its Natural Language Understanding Service.

By the way, if you are interested in learning about the differences between “sentiment” and “emotion“, I encourage you to go through this article and this one.

The set of emotions detected by different APIs varies somewhat. IBM, for instance, detects:

The standard reference for the set of emotions is the work by Paul Ekman. This Wikipedia article gives interesting details on various emotion categories.

Getting back to today’s experiment, I decided to use IBM’s Python SDK instead of directly accessing their HTTPS end point.

The program itself is trivial:

Program to Dump Emotions

The first sentence to try is:

“It was scary to drive alone on the highway.”

Here is the output from the program:

Identified Emotions

This shows that “fear” is the primary emotion. I am sure everyone will agree.

Here is the second sentence:

“I can’t wait to see the President in person!”

The program identifies the following emotion weightage:

Identified Emotions

In this case, the main emotion is “Joy“. Although this is as expected, I was hoping for a greater weightage for “Joy“, say, more than 0.7. It is not clear why it is just 0.32.

The next sentence is:

“The talk was dull and uninteresting. The audience was literally yawning throughout the program.”

Let us see how the program determines the emotions:

Identified Emotions

Interesting. “sadness” and “disgust” predominate, with almost equal weightage. This seems OK to me.

Let us try this one:

“Our whole family rejoiced when my son got the first prize.”

Obviously, this is a good example of “Joy” being the primary emotion. What does the program have to say?

Identified Emotions

Great. The program’s output agrees with our intuition.

Here is our last example:

“I don’t understand why our politicians are so arrogant.”

The output for this is:

Identified Emotions

So, “disgust” and “anger” predominate. Couldn’t agree more!

Using the SDK itself is quite straightforward, and with 30000 free NLU items per month, this service is definitely worth checking out. Next week, I plan to look at ParallelDots API for detecting emotions.

Bye, until then! Have a fantastic weekend!

]]>https://www.rangakrish.com/index.php/2019/02/17/identifying-emotions-from-text/feed/01460Coreference Resolution in Stanford CoreNLPhttps://www.rangakrish.com/index.php/2019/02/10/coreference-resolution-in-stanford-corenlp/
https://www.rangakrish.com/index.php/2019/02/10/coreference-resolution-in-stanford-corenlp/#respondSun, 10 Feb 2019 08:34:34 +0000https://www.rangakrish.com/?p=1444In the last article, I showed how we can use the neuralcoref library along with spaCy to do coreference resolution (examples involved anaphoric references). In today’s article, I want to try the same (well, almost) examples in Stanford CoreNLP engine and see how they compare.

Since CoreNLP is a Java implementation, I chose to write the test program in Java. Here is the programalong with the examples.

Strange. It shows that the program has identified “My sister” and “she” as pointing to the same entity, but there is no mention of “dog” and “him”. In contrast, neuralcoref identified both the “mentions” correctly.

Could it be because this is a compound sentence? Let us convert this into two simple sentences:

“My sister has a dog. She loves him.”

Here is the output in this case:

Example-2 Output

OK! It is interesting that the program has identified both the “mentions” this time.

I am using the remaining sentences as they were in neuralcoref example. Here is the third sentence:

“My sister has a dog and she loves him. He is cute.”

The output is:

Example-3 Output

Although the system has identified “him” and “he” as belonging together, it has omitted “dog” from the reference. Bug! Here also, neuralcoref fared better.

The next example is:

“My sister has a dog and she loves her.”

The output is:

Example-4 Output

The behavior is the same as with neuralcoref. “dog” is not paired with “her”.

Consider the following statement that uses different genders and hence should be easier to resolve:

“My brother has a dog and he loves her.”

This is what we get:

Example-5 Output

That is nice! “dog” and “her” are correctly paired. neuralcoref did not handle this case correctly.

Let us look at the next sentence:

“Mary and Julie are sisters. They love chocolates.”

neuralcoref handled this case correctly. Strangely, when given this input, CoreNLP does not identify any “mentions” at all! I confess I am disappointed.

Let us try the next one:

“John and Mary are neighbours. She admires him because he works hard.”

This looks more complex than the earlier one. How does CoreNLP fare?

Example-7 Output

OK, this one is handled correctly. That is a relief.

The next one is tougher:

“X and Y are neighbours. She admires him because he works hard.”

Here is the program’s output:

Example-8 Output

Not surprised with the behavior. neuralcoref wasn’t any better.

Here is the last example:

“The dog chased the cat. But it escaped.”

This is the corresponding output:

Example-9 Output

Not correct. Again, the behavior is identical to neuralcoref.

Let me summarize and show the performance of CoreNLP and neuralcoref with respect to the examples:

Comparison Table

You can see that among the sentences tested in both the systems, neuralcoref got 4 out of 8 correct, whereas CoreNLP got just 2 out of 8 correct. To clarify, I have marked the output as “Wrong” if the system did not FULLY identify the mentions.

Overall, in the case of anaphora resolution, both the popular libraries have fared poorly in my opinion. I expected more out of CoreNLP, so that was a bigger disappointment!

Looks like there is a lot more work to do in the area of coreference resolution!

]]>https://www.rangakrish.com/index.php/2019/02/10/coreference-resolution-in-stanford-corenlp/feed/01444Coreference Resolution Using spaCyhttps://www.rangakrish.com/index.php/2019/02/03/coreference-resolution-using-spacy/
https://www.rangakrish.com/index.php/2019/02/03/coreference-resolution-using-spacy/#respondSun, 03 Feb 2019 10:30:20 +0000https://www.rangakrish.com/?p=1427According to Stanford NLP Group, “Coreference resolution is the task of finding all expressions that refer to the same entity in a text”.You can also read this Wikipedia page.

For example, in the sentence “Tom dropped the glass jar by accident and broke it”, what does “it” refer to? I am sure you will immediately say that “it” refers to “the glass jar”. This is a simple example of coreference resolution.

It can be much more tricky in some cases, but humans usually have no difficulty in resolving coreferences. In today’s article, I want to take a look at the “neuralcoref“ Python library that is integrated into spaCy’s NLP pipeline and hence seamlessly extends spaCy. You may also want to read this article.

Installing the library is simple. Just follow the instructions given here. I have heard that it does not work well some versions of spaCy, but my version of spaCy (ver 2.0.13) had no compatibility issues.

We need to import spaCy and load the relevant coreference model. I chose the “large” model just to be safe.

Loading the Coreference Model

I wrote two simple functions, one to print all the coreference “mentions” in the document, and another to print the resolved pronoun references.

In the above examples, we implicitly assumed that the dog is male. What if we use the pronoun “her” instead of “him”, implying the dog is a female?

“My sister has a dog and she loves her.”

This is the output we get in this case:

Output

Clearly, the mapping of “her” to “My sister” is not what we expect. Could it be because of some confusion caused by both objects being females? Let us try a modified example:

“My brother has a dog and he loves her.”

In this case, since “brother” refers to a male, we expect that “her” should be easily resolved to the only other object, “dog”. Let us see:

Output

Strange! Even in this case, the library incorrectly maps “her” to “My brother”. Such errors could be due to the model, or because the machine learning approach does not lead to “understanding” in the way we humans understand text.

Let us try another example:

“Mary and Julie are sisters. They love chocolates.“

In this case, we expect “they” to refer to both “Mary and Julie” and not just one of them. Here is the output:

Output

That is nice. Works as expected. Let us introduce a twist in this pattern:

“John and Mary are neighbours. She admires him because he works hard.“

We know that “John” is male and “Mary” is female, so “She” should map to “Mary” and both “him” and “he” should point to “John”. How does the library handle this?

Output

That is a pleasant surprise! The system resolved the pronouns correctly.

Wait. What if we use abstract names such as “X” and “Y” instead of “John” and “Mary”?

“X and Y are neighbours. She admires him because he works hard.“

This is what the library does in this case:

Output

Let us think for a minute here. How would we, as humans, have handled this case? I feel it is acceptable if “She” maps to “X” and both “him” and “he” map to “Y”, or the other way around. But the way the library has resolved the references stumps me!

Let us try one last example:

“The dog chased the cat. But it escaped.”

This is what we get:

Output

I think we, as humans, would have mapped “it” to the “cat” without much effort, because we understand the context. Erroneous mappings from the system appear unavoidable, given that the system does not “understand” the meaning of the sentence. As I mentioned earlier in thearticle, Coreference resolution is a complex task and I expect that neuralcoref library and other similar systems will become better in due course.

In case you haven’t noticed, all the examples I have considered for this article involve one type of coreference called “anaphora”. There are other types, which can be even more difficult to handle.

It would be interesting to compare the performance of other libraries such as OpenNLP and Stanford Parser on the same set of examples. Well, that is for another article.

]]>https://www.rangakrish.com/index.php/2019/02/03/coreference-resolution-using-spacy/feed/01427Generating Poetry Using iLangGenhttps://www.rangakrish.com/index.php/2019/01/27/generating-poetry-using-ilanggen/
https://www.rangakrish.com/index.php/2019/01/27/generating-poetry-using-ilanggen/#respondSun, 27 Jan 2019 02:07:19 +0000https://www.rangakrish.com/?p=1410In an earlier article, I wrote about using iLangGen to generate natural language text. iLangGen is a powerful text generation library that I have been working on over the years. Today, I would like to show how we can use that library to generate “poetry“. Be warned, however, that the generated poem is devoid of meaning and coherence, similar to what I had generated with the RiTa library. The idea is to show the kind of interesting things that one can do with iLangGen.

First, we need to get words with interesting patterns and properties, for example, “adjectives” that end in “ate“, or “verbs” with 4 syllables, and so on. For this I use my iLexicon as backend.

Here is how we set up the words to use:

Defining Word Lists

The LHS is a variable that gets bound to the evaluated expression on the RHS.

The next step is to define one or more grammars that control the generation process. Here is a simple version of “Verb Phrase“:

Grammar for Verb Phrase

You will notice that in some of the productions, the RHS involves invoking a function to dynamically fetch a word from the word list. For instance, the production with “adjective” as the LHS gets a random adjective from our word list.

The last production in “VerbPhrase” defines a “word separator“, which is itself defined in another grammar called “Utils“. This ability to reference productions in other grammars is an example of grammar re-use in iLangGen and allows us to develop complex grammars. I have described the features of iLangGen in a series of articles earlier (August 2017). Here is the simple “Utils” grammar:

Grammar: Utils

To make the generated poem look more realistic, we define a few more grammars:

More Grammars

To add further variety, let us define two grammars to generate simple “why” and “who” questions:

Grammars for Questions

The third step in the generation strategy is to define “sentence patterns“. Each line (or sentence) in the poem is generated using this pattern. The pattern itself refers to a grammar and one or more non-terminals in that grammar. The generator synthesizes a line of the poem from the referenced grammar, randomly choosing a non-terminal from the list. Here is how we define the patterns:

Defining Sentence Patterns

When the generator decides to generate a line according to “pat3“, for instance, it will use “AdjPhrase” grammar and one of its non-terminals “phrase3“,“phrase5“,or “phrase6” at random.

Finally, to generate a poem, we use the “generate-poem” function, supplying a list of patterns. Here is a sample output:

Sample Output 1

Each group of patterns defines a “stanza” or “para“.

If we execute the function again, passing the same set of arguments as patterns, we are guaranteed to get a new poem. The text is not repeated!

Sample Output 2

Here is a simpler version using just the pattern “pat1“:

With a Single Pattern

As you can see, generating text of arbitrary nature, including poems, is possible with iLangGen. Of course, to get acceptable quality, considerable effort needs to be put in in terms of the grammar and choice of words. For example, take a look at “Sent” grammar. It is used in “pat6” and is capable of generating sentences that rhyme by using words that end in “ate“. For this, the generator is ably assisted by the other core component, “iLexicon“.

Hope you enjoyed reading this post. Have a nice weekend.

]]>https://www.rangakrish.com/index.php/2019/01/27/generating-poetry-using-ilanggen/feed/01410Named Entity Recognition (NER) with OpenNLPhttps://www.rangakrish.com/index.php/2019/01/20/named-entity-recognition-ner-with-opennlp/
https://www.rangakrish.com/index.php/2019/01/20/named-entity-recognition-ner-with-opennlp/#respondSun, 20 Jan 2019 03:48:26 +0000https://www.rangakrish.com/?p=1399In the earlier two articles, we looked at Sentence Parsing and Chunking as supported in OpenNLP. In today’s article, let us explore Named Entity Recognition, also known as NER.

NER is a technique to identify special categories of noun phrases such as people, places, companies, money, etc., present in the given text. This is widely used as part of information extraction. Here is a nice Youtube video on NER.

The two primary classes that are used for named entity recognition in OpenNLP are

TokenNameFinderModel

TokenNameFinderME

The former constructs a model from a model file, and the latter uses the model for entity recognition.

OpenNLP supports NER of the following categories:

People

Location

Organization

Date

Time

Percentage

Money

Quite broad, I would say.

For the purpose of this article, I am going to work with the first three, namely, People, Location and Organization. It is easy to extend my sample code to include the others too, if you want.

First, we have to download the relevant model files. Here is a table that shows the model files I am using in my code:

People en-ner-person.bin

Location en-ner-location.bin

Organization en-ner-organization.bin

The following code snippet shows a sample session, working on three sentences.

Sample Session

The output is here:

Output

You can see that it is possible to get the entities as well the associated probabilities when using OpenNLP’sNER logic. I have defined two wrapper classes to make it easy to work with OpenNLP. Ideally, they should be moved to separate source files and made public, but for ease of demonstration, I put them all in the same file (and hence they are non-public).

Here are the wrapper classes:

Wrapper Classes

It is quite easy to include support for the other named entities not covered in this example. You can download the source code from here.

Have a nice weekend!

]]>https://www.rangakrish.com/index.php/2019/01/20/named-entity-recognition-ner-with-opennlp/feed/01399Chunking in OpenNLPhttps://www.rangakrish.com/index.php/2019/01/13/chunking-in-opennlp/
https://www.rangakrish.com/index.php/2019/01/13/chunking-in-opennlp/#respondSun, 13 Jan 2019 08:42:49 +0000https://www.rangakrish.com/?p=1386In my previous post, I showed how to parse sentences using OpenNLP. Another useful feature supported by OpenNLP is “chunking”. That is the subject of today’s article.

Chunking stands between part-of-speech tagging and full parse in terms of the information it captures. POS tagging assigns part of speech to individual tokens in a sentence. So, in the sentence “Peter likes sweets”, the POS tags are:

The constituency parser operates at the other extreme. It tries to assign a structure to the complete sentence, by assigning a structure (recursively) to constituent parts. We saw this in the last article.

Full parse is significantly more expensive than just POS tagging for obvious reasons. Sometimes we might be interested only in the smaller structures contained in the larger parse tree, for example, Verb Phrase, Adjective Phrase, Noun Phrase, and so on. The classic example is NER (Named Entity Recognition) where we are interested in specific Noun Phrases. This usually (not always) involves more than one token in the given text, and is called “chunking”.

OK. Let us see how to use the chunker in OpenNLP. I have written a simple class called “OpenNLPChunkerExample” to illustrate the essential features (you can download the source from here).

The code fragment below gets the chunked tags and prints them along with the corresponding word.

Printing Chunked Tags

The output from the program is:

Output

The tagging produced by the chunker follows the “IOB” tagging scheme. Here,

B = Beginning of chunk

I = In a chunk

O = Outside any chunk

From the above scheme, we can easily see that the words “The pretty cat” form a single NP chunk, the word “chased” forms a VP chunk all by itself, and the words “the ugly rat” constitute an NP chunk again. The final “.” is not part of any chunk.

To facilitate readability, we can write a convenience function to group the related chunks. Here is the code:

Function to Group Words in Chunk

The function returns a Span[]. The updated “main” that uses this function and prints the chunks is:

Printing Grouped Chunks

The corresponding output is:

Grouped Chunks

We can even get the probability associated with each chunked tag. Here is the final version that prints this information:

Printing Chunked Tags with Probability

Here is the corresponding output:

Tags with Probability

Before concluding, let us print the chunks for another sentence: “It is very beautiful.”

Another Example

You can see that we now have an Adjective Phrase (ADJP): “very beautiful”.

Python’s NLTK, another popular NLP toolkit, also supports chunking. What I like about NLTK is that it allows us to define a “chunking grammar” to customize our chunking logic. This can prove useful in some cases. Take a look at NLTK when you get time.