Monday, May 19, 2014

The Fog is an extension of the Cloud. So in Fog Computing, data is stored closer to the devices we use to access the internet. The benefit is a reduction of bandwidth and latency.

CISCO routers already live in this grey area between out devices and big data providers. CISCO is adding Linux operating systems to some of their routers, and manage frequently accessed data with a distributed network of these router on the edge of the cloud.

Tuesday, December 24, 2013

This article provides an underlying structure for managing the big data phenomenon. Innovations and tools fundamental to handling big data are highlighted, and we look at how these technologies are being implemented in the financial industry.

It's a favorite exhibit of those who have read the paper. Many had never heard of terms larger than a petabyte. In the article, I mention that the term yottabyte is used in "speculation". But is that speculation about to enter the realm of reality? The recent article about Twitter adding security to impede surveillance mentions that the National Security Agency's datacenter in Bluffdale, Utah is "possibly cabable" of storing a yottabyte. We're still in speculation mode but for how long? Will 2014 be the year of the Yottabyte?

Tuesday, September 3, 2013

Still mass marketing potential customers with emails? You may instead be driving them away. Aside from being technologically dated, unsolicited emails, far more than likely, never get read, leaving only a brief unprofessional impression.

MailChimp

Some of you might be thinking, “Great, now you can blast those people press releases!” If so, please unplug your computer now. I can think of no better way to lose a customer than to “blast” them with–well, anything. - Ben Chestnut MailChimp CEO

Instead of driving customers away, MailChimp allows users to employ basic marketing techniques such as segmentation with their subscriber lists. The Discovery feature uses big data techniques to parse millions of subscribers in minutes, and help find new customers.

Friday, August 2, 2013

Visualizations

Obtaining insights from Big Data requires new ways of looking at data. One of the most popular visualizations is the Tag Cloud.

The Tag Cloud is a creative way to convey lots of information about single articles, large websites, or even entire libraries, all on a single screen. Words are sized relative to the number of times they appear, and clustered together randomly to give a reader a different perspective. This Tag cloud contains the most common words on the AppTrain blog homepage.

A Mind Map

is a Tag Cloud with relationships. Rather than random words, a Mind Map displays key concepts and how they relate to each other.

is a javascript library for building visualizations from any dataset. The libraries examples are an extensive collection of modern visualizations. Many alternate ways of adding dimensions to graphs are displayed on the example page including Voronoi Diagrams, Box Plots, Heat Maps, Bubble Charts,
Van Wijk Visualizations, Hamiltonian Graphs; Many of the examples are also interactive, changing the way the graph is viewed based on user input. Exploring the examples is truly mind expanding.

This graph shows the DJIA on the left, and color codes 3 week periods in the graph according to search frequencies of the word debt. Note that red weeks, like late October of 2008 correspond with declines in the DJIA. So when lot's of people were searching for the work debt, the stock market went down.

The word Debt was the best performing term in the study. Notably , it performed better than terms like nyse ,nasdq, and dow jones.

The researchers then compared a Google Trends investment strategy to a basic Buy and Hold strategy and a random Dow Jones strategy.

The results were remarkable. The Google Trends strategy far exceeded the other strategies.

"To put it in perspective, I was recently invited to attend an event on algorithmic trading hosted by Kissell Research Group. The last presentation by Michael Blake was on Smart Technology for Big Data." ...more

Wednesday, March 27, 2013

This 1998 SiliconGraphics ad from Black Enterprise magazine offers solutions for a "Big Data" world. 256GB of system memory on a server and 400 Terabytes of storage. Not bad for the 20th century. Or for this century.

Weiss goes on to say "While big data have the potential for better results, there is no guarantee that they are more predictive than small data" With all the hype around Big Data, it helps to get back to the origins of the term and realize that it's one of may interesting problems that experts in a variety of disciplines have been wrestling with for a long time.