Archive for the ‘Linux’ category

The New York Times is running an interesting piece about the ever growing glut of data. The article details IBM and Google’s concern over the data glut and if new and upcoming students trained to handle the explosion of data. It is quite a fascinating piece.

At the heart of this criticism is data. Researchers and workers in fields as diverse as bio-technology, astronomy and computer science will soon find themselves overwhelmed with information. Better telescopes and genome sequencers are as much to blame for this data glut as are faster computers and bigger hard drives.

Please click through and read the whole article. It is very good and very true. This topic should be at the forefront of any person who works in the Computer/Technology field. First there is the problem of how to store this much data. Currently I work for a small publisher (O’Reilly media). It is easy to think that a small publisher probably doesn’t have huge storage needs. But so far since I’ve started working here (1 full year going on my second) we just ordered our second storage shelf, this time for almost 14TB. The new shelf has yet to be installed, but the other day my IT coworker was talking to management in a meeting. Our last shelf was around 1TB, but lasted less than a year. He said at almost 14TB this should last us a long time, but then added, “But we say this every time.” It is so true, especially with storage so cheap and drives so big. It reminds me of my first computer in the mid 90’s with 10GB of storage. I told my parents I’d never needed a bigger hard drive. Then I went away for my freshmen year of college and filled it right up with stupid pictures and movie files.

When I worked for the University of Illinois Engineering department the problems were worse. One research group that I worked for had 1 professor and maybe 5 students (including undergrad). They were relatively new so there was no infrastructure or file server and there really wasn’t much money for it anyway. One day I went to the Professor’s office. He must have had at least 30 hard drives each at least 500GB if not 1TB. Those were just the hard drives he had his students carried around a handful themselves. Another research group, with decades of history, started a scanning project. They would scan hundreds of slides at once each producing around 1MB of data. We installed a file array starting off at 4TB, but was expandable to 14. Unfortunately I left and am not sure what they have or need now. My point is that data storage is a huge problem. And is growing extremely fast. The article mentioned facebook’s 1Petabyte of photos, I’m guilty of quite a few of those, but that is just mentioning one company, many more could have been mentioned. Finally there is even personal space. Since I got my new camera I myself am looking at more storage for home. I am looking for personal NAS boxes. So I see the basic point. The future of IT is data and what to do with it.

Computer scientists and, for that matter, any scientists need to pay special attention. Not only do we need a way to store a lot of this data, but probably more importantly we need to do something with it. A lot of this will rest on programmers, but it isn’t limited to them. When I worked at the U of I the students worked on a cluster I built for them. They would code in C tweaking their algorithm to save every last processor cycle. These students weren’t in Computer Science. This summer I took a course at Boston University. One of my classmates was clearly not a computer person. I asked her why she took the course. She was a statistician and was heading to Grad School for statistics. The school asked her to take programming courses so she could analyze data sets. And of course then there are the Computer Scientists, and our future depends upon analyzing such data.

The future is big data; lots of it. And it is no longer just Google and IBM analyzing and storing it. Now even the smallest of research groups or a little publisher can generating mounds of information. Time to start paying very close attention.

There has been a controversy brewing for quite some time in the Open Source world. I’ve sort of stayed out of it because I didn’t fully understand exactly what was happening, and I wasn’t sure that I cared. Still there is good reasons if you are a programmer looking to do anything in the non-MS world with C# to at least be aware of this. I currently don’t have any C# training, but I am scheduled to take a class on it in the Spring. Finally I found an explanation that didn’t resort to needless flame wars. I think it is fair and balanced. Please click through, but I’ll post a few highlights. So you know mono is an open source project that ports .net framework to Linux and Mac. So this is where the controversy lies. It is porting over a Microsoft technology, and MS does not have a good track record with Linux or open source. Also, by way of background, there are a few very notable programs that are written with mono and are being considered as default programs in Linux distributions (tomboy, banshee, fspot, and gome-do).

Mono, the free software implementation of .NET (C#), has been the subject of bitter debate for eight years. Yesterday, that debate ended — or at least shifted to another level — with Microsoft’s announcement that it was extending its Community Promise to include the patents that left Mono possibly encumbered.

…

The greatest fear has been that Mono-based programs like GNOME’s Tomboy or F-Spot could be the source of a patent violation case by Microsoft against some or all of the community.

In 2001, Microsoft released a letter to ECMA in which it promised that use of the patents involved would be available on request on a “royalty free and otherwise RAND ‘Reasonable and Non-Discriminatory’ basis.”

However, as Miguel de Izaca, the founder of Mono and a Novell vice president, points out, “The problem with ‘RAND’ is that it doesn’t say what ‘reasonable’ means. It has to be reasonable, but it doesn’t have to be free. Microsoft stated publicly and on the ECMA committee that nobody had to pay, but they never actually went and published the license.”

And there is the problem. While C# looks like a great language with awesome capabilities the fact that MS holds patents and is a commercial entity leaves the door wide open to forcing Linux users to pay up.

As described on his blog, de Izaca plans to divided Mono source code into two repositories. One will include the ECMA-covered libraries, and the other Mono’s implementation of ASP.NET, ADO.NET, and Winforms. By making this division, de Icaza presumably hopes to make clear to developers at a glance what code they are working with.

I’m excited to learn C# and really have every intention to use it, but this does give me pause especially if I were starting a large project that needed mono (i.e for any platform other than Windows).

Microsoft recently released code (some 20,000 lines) for inclusion in the Linux kernel. However, look a bit deeper… They’re all about the M$. So it has to be capitalistic. Keep reading and you find that the drivers make it possible to run Linux on HyperV, MS’s new toy in the server market. Here’s a good summary of how it looks right now:

HTC the cell phone maker was the first company to come out with an Android phone with the G1. They now have two more announced. This latest one has no certain date in North America, but it looks good. I think I might want one.

*
Up to 750 hours for WCDMA
*
Up to 440 hours for GSM
microSD™ memory card (SD 2.0 compatible)

This is certainly going to be a compelling phone. It is no secret that I like Android. This added to the their previous phone leads me to believe that Android could take Apple’s iPhoneOS to task. I’m thinking about jumping into mobile development and Android is looking way better than Apple right now to me.

This would be quite welcomed by me. Ars is reporting that up to 20 Android phones may be released by the end of the year. I am a huge fan of Android…at least I want it to succeed. Currently I am an iPhone owner, but honestly if the G1 had been a little compelling at the time I might have switched. I am hoping that as more Android phones are released more developers will create compelling apps for the system. There is no word on the “who, what, when, or where,” just yet. The article seems to suggest at the end that this may be more over in Europe with the America following next year:

He also pointed out that, because of the highly competitive handset market in the US, carriers here wouldn’t be as likely as those in Europe to jump on the new devices unless they are distinctive. That means Android fans in the US who aren’t keen on switching to T-Mobile for the G1 may have to wait a little longer than our friends on the other side of the pond, but if Google is on a roll with its distribution agreements, it undoubtedly won’t be long before more phones pop up.

I think it is a bit of a misnomer to say that we have a “highly competitive handset market.” Mainly because that would suggest that other economies don’t. The reality is that we have a highly competitive service provider market who artificially create scarcities to try to lure customers to their service. ATT makes pennies off of the sale of an iPhone, but the data charges and 2 year contract mixed with the sexy appeal of being the only iPhone provider give it a competitive advantage. There are people who would die to get an iPhone but won’t touch ATT. In other countries, particularly Asia, you purchase your phone first and get a service provider second. I think it is fair to say that they have a much more competitive handset market where we have a “highly competitive” service provider market. in Asia the handset makers actually have to do marketing work to lure you into a purchase. Personally I like the Asian way. Back when cell phones were brand new technology it made sense to subsidize phones based upon a 2 year contract. People were skeptical of the service already fearing high prices, and cell phones that comparatively do little cost exorbitant amounts of money. At this point, however, I would be more than willing to purchase a phone outright to avoid a contract. In theory this is possible (though not with an iPhone), but in practicality it isn’t. Given the two major carriers, verizon and ATT, a phone purchased for one would not necessarily transfer to the other giving little reason not to subsidize your phone.

So other than that slight tangent I believe this is good news. I think my timing will work out just right that when my iPhone contract ends I’ll have a couple of compelling choices to choose from as an alternative.

I haven’t had too much experience with the G1 and I thought it was a little bit of a dude, but I am still very excited about the Android project. The potential is there and if someone could pair a good phone with the nice OS I think the market would boom. I do, however, think this latest bit of news is rather exciting. Ars is reporting that Canonical developers aim to make Android apps run on Ubuntu. From the article:

Canonical is building an Android execution environment that will make it possible for Android applications to run on Ubuntu and potentially other conventional Linux distributions. The effort will open the door for bringing Android’s growing ecosystem of third-party software to the desktop.

Google’s Linux-based Android platform is attracting a lot of attention. The new version significantly improves the platform’s reliability and could make it look a lot more appealing to carriers and handset makers. The availability of an experimental x86 port has caused some people to speculate that Android might have a place in the netbook market.

Count me in. This is exciting news. Hopefully we’ll see Android take off. I know I am rooting for it.

Ubuntu released its latest version 9.04 codenamed Jaunty Jackalope. Remember that Ubuntu releases on a six month rotation, but have a two year cycle for LTS (Long Term Support) versions. This new version looks like a good step forward for the development, but if you are using Ubuntu in production services I would keep with 8.04LTS. Ars has a good rundown of the new features. Of particular interest to me is the Netbook Remix version. I have been using the prerelease of this on 8.10. I think this is a bold new step to get Ubuntu specifically tailored for the netbook and I like the design a lot.

As always Ubuntu Servers are bogged down. Expect huge delays and dropped packets if you do the upgrade over the web. One easy way around this is to choose mirrors other than the default (MIT’s is usually running pretty fast). I may drudge up the instructions for upgrading over bittorrent as well.