Swivel Puts a New Twist on BI for the Masses
by Brian Moran, brian@solidqualitylearning.com

This week’s commentary is a bit non-traditional; it’s all about data, but has little to do with SQL Server. My thoughts this week were sparked by an email message that one of my business colleagues sent with the subject line “This could put ‘BI for the masses’ in a new light.” I’ve always been a big fan of the concept of BI for the masses and have written about the idea many times because I think that it will happen one day. So a message with that subject line is bound to get my attention.

Calling Swivel “The internet archive for data” and saying it’s a “bit like YouTube for data” might sound like lofty aspirations for a site that is just in the preview stages. In fact, I had a hard time justifying to myself the need to write about Swivel before anyone has evaluated whether the service really works. But it’s such an interesting idea that I simply couldn’t resist thinking out loud about the potential effect. To me, the idea of “BI for the masses” has always implied more of a corporate slant. I never really expected Great Aunt Suzzie to be “doing BI.” But, what if BI, data-mining—or whatever fancy techno-terms we use to describe visual data analysis—does become truly mainstream, adopted by the masses?

The article quotes an unidentified PhD student working in bioinformatics as saying, “For me this sounds potentially very interesting. It would be useful to share data for collaborative research, especially if they make it easy to access the data via APIs. I could even release alongside a publication the means to fully reproduce the calculations via this site. Others could quickly build on the publication with access to the data and analysis.”

Ok, so I don’t think Swivel is going to put corporate IT out of work anytime soon. The data privacy and security problems are simply too complex to deal with. But what if something like Swivel made it easy for people who aren’t functional experts to “do BI” without dedicated IT support? What if these unwashed masses, lacking the benefit of corporate IT, could publish, analyze, and understand complex data sets in a holistic interrelated manner? Might that create a vast, almost unimaginable, increase in the level of complex data analysis that happens worldwide? Might it work in almost the same way that the simple Internet browser coupled with search engines creates a vast encyclopedic collection of content that includes random junk as well as exactly the right thing you were looking for and that would have otherwise been impossible to find? Interesting.

I also wonder whether something like Swivel will be part of the ultimate solution that unlocks the data and insight stored in the world’s spreadmarts. Or maybe this is simply a neat idea that will never even get off the ground. Swivel might seem like a silly idea to hard-core data professionals, folks who would wonder and doubt about the accuracy of any data captured and published in such a non-controlled manner. I share those doubts as well. But then again, I was pretty surprised when Google bought YouTube for a zillion trillion dollars (or what ever the crazy, unimaginable number was). I have a much easier time seeing Google being interested in an idea like Swivel as part of a comprehensive, long-term plan to create a robust data-analysis empire. I’m just sad that I didn’t think of it first.

Ensure Continuous Data Uptime
GoldenGate Software announced GoldenGate 9.0, software that improves the availability, accessibility, and performance of a company's mission-critical data. GoldenGate extends its heterogeneous database support to include SQL Server 2005 and IBM DB2 z/OS. GoldenGate works by moving data from the source to target in real time without a middle tier. The software reads the SQL Server database log, encrypts the committed transactions, and moves them across the network to one or many targeted systems. To ensure continuous uptime to your data, GoldenGate moves the data bi-directionally while maintaining transaction integrity, and during a primary system outage, users are pointed to a second fully synchronized system. For more information, contact GoldenGate Software at 415-777-0200 or visit http://www.goldengate.com.

Defragment SQL Server Indexes
Idera announced SQL defrag manager, an automated analysis and defragmentation solution for SQL Server databases. The software improves SQL Server database performance by continuously analyzing fragmentation levels and automatically defragmenting indexes in which performance has eroded. SQL defrag manager's management console gives you a real-time window into fragmentation levels, and can manage and control automated analysis and defragmentation activity across hundreds of servers and thousands of databases. SQL defrag manager determines the best-fit approach and schedule for defragmentation to ensure optimal results with the least possible effect on critical production servers. The software also provides reporting on the analysis and defragmentation activity. For more information, contact Idera at 713-523-4433 or 877-464-3372 or visit http://www.idera.com.

Sponsor: InnovartisISV? Database Version Upgrades Painful? DB Ghost Packager Plus enables ISVs to develop an installer for their customers containing the new version of their product that will automatically bring any target database to the same level as the source database. ISV customers just launch the Packager Plus installer.exe, choose the location and name of the database, set the basic settings, and click—it is done. Excuse the cliche, but it really is as simple as that. Download our 14-day Free Trial.

3. Hot Articles

Reader to Reader: Summarize All the Constraints for a Table
Send your SQL Server code, comments, discoveries, and solutions to r2r@sqlmag.com.

Maximize your investment in your VoIP network by using all of its capabilities. Learn to integrate Fax for IP to reduce the TCO and increase ROI for your investment. On-Demand Web Seminar.

Discover a wealth of information about how to protect and secure your data in the event of a disaster. You may not be able to predict the exact details of a disaster, but you can be prepared with a solid response for when one strikes. Disaster can strike anywhere—not just where severe weather can hit—so make sure you’re ready when it does. Download your free copy of this eBook today!

Does your SQL Server deployment suffer from low utilization rates, sub-optimal availability, and management challenges? Learn to overcome these difficulties by deploying a single database utility that can help lower your cost of ownership by 70% and react to changes within 30 seconds. On-Demand Web Seminar.

Got information? Good! Now, can you use it? Get up to speed on database design and hierarchy, including columns and data types, creating databases, and using the Query Editor. Download your free copy today!

The average enterprise spends nearly $10 million annually on IT compliance. Download this free white paper today to streamline the compliance lifecycle, and dramatically reduce your company’s costs!

Bonus: Register for any whitepaper from Windows IT Pro during December — you could win a Nintendo Wii! View the full whitepaper listing at http://www.windowsitpro.com/whitepapers — and remember, the more you download, the better your chances of winning.

Congratulations to John T. Keller, who won first prize of $100 for the best solution to the December Reader Challenge, "Improve T-SQL to Prevent Deadlocks" and to Bill Twomey who won second prize of $50. John is a database developer for Softek Solutions, and Bill is a SQL Server DBA for Database Solutions. You can read a recap of the problem and the solution to the December Reader Challenge at http://www.sqlmag.com/Article/ArticleID/94572/sql_server_94572.html.

January Reader Challenge: Converting a Hexadecimal String to a Varbinary Value
Now, test your SQL Server savvy in the January Reader Challenge, "Converting a Hexadecimal String to a Varbinary Value" (below). Submit your solution in an email message to challenge@sqlmag.com by December 21. Umachandar Jayachandran, a SQL Server Magazine technical editor, will evaluate the responses. We'll announce the winner in an upcoming SQL Server Magazine UPDATE. The first-place winner will receive $100, and the second-place winner will receive $50.

Here's the challenge:
Bart is a DBA who maintains several data warehouses, which are hosted on SQL Server 2000. One data warehouse has a table called ExtFeeds that periodically receives data as flat files loaded from an external source. One of the columns in the flat file has a hexadecimal string value, ranging from 4 to 32 bytes, which is stored in the ExtFeeds table's DataHdr column. The external source can't be modified easily, and other applications rely on the DataHdr values to be strings, so Bart can't manipulate the value or convert to binary data. Instead he loads the files directly into the table by using the bcp utility.

However, Bart needs to periodically write queries against the ExtFeeds table that involve accessing the DataHdr column. The queries perform various operations such as converting the hexadecimal string value to varbinary and joining the ExtFeeds table with other tables. Help Bart design a solution that helps convert the hexadecimal strings to varbinary values. He would need to perform the conversion in the SELECT list of queries, in the logic in stored procedures, and in INSERT and UPDATE statements. The solution should be able to support values ranging from 4 to 32 bytes and perform efficiently.

Holiday Offer—Save $40 on SQL Server Magazine
Don't miss SQL Server Magazine in 2007! As a subscriber, you’ll have full access to must-have content covering high availability, SQL Server and Microsoft Office integration, business intelligence, clustering, Reporting Services and much more. Order now and save $40: https://store.pentontech.com/index.cfm?s=1&promocode=eu216cuw.

Make Your Mark on the IT Community!
Nominate yourself or a peer to become an "IT Pro of the Month." This is your chance to get the recognition you deserve! Winners will receive more than $600 in IT resources and be featured in Windows IT Pro magazine and the TechNet Flash email newsletter. It's easy to enter—accepting January nominations now for a limited time! Submit your nomination today: http://www.windowsitpro.com/go/itpromonth.

SQL Server Magazine UPDATE is brought to you by SQL Server Magazine, the only magazine devoted to helping developers and DBAs master new and emerging SQL Server technologies and issues. Subscribe today!