Search results matching tags 'Azure' and 'Azure Use Cases'http://sqlblog.com/search/SearchResults.aspx?o=DateDescending&tag=Azure,Azure+Use+Cases&orTags=0Search results matching tags 'Azure' and 'Azure Use Cases'en-USCommunityServer 2.1 SP2 (Build: 61129.1)Keeping your options open in a cloud solutionhttp://sqlblog.com/blogs/buck_woody/archive/2012/09/25/keeping-your-options-open-in-a-cloud-solution.aspxTue, 25 Sep 2012 14:20:00 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:45345BuckWoody<p>In on-premises solutions we have the full range of options open for a given computing solution &ndash; but we don&rsquo;t always take advantage of them, for multiple reasons. Data goes in a Relational Database Management System, files go on a share, and e-mail goes to the Exchange server.</p>
<p>Over time, vendors (including ourselves) add in functionality to one product that allow non-standard use of the platform. For example, SQL Server (and Oracle, and others) allow large binary storage in or through the system &ndash; something not originally intended for an RDBMS to handle. There are certainly times when this makes sense, of course, but often these platform hammers turn every problem into a nail. It can make us &ldquo;lazy&rdquo; in our design &ndash; we sometimes don&rsquo;t take the time to learn another architecture because the one we&rsquo;ve spent so much time with can handle what we want to do.</p>
<p>But there&rsquo;s a distinct danger here. In nature, when a population shares too many of the same traits, it can cause a complete collapse if a situation exploits a weakness shared by that population. The same is true with not using the right<br />tool for the job in a computing environment. Your company or organization depends on your knowledge as a professional to select the best mix of supportable, flexible, cost-effective technologies to solve their problems, whether you&rsquo;re in an architect role or not.&nbsp;</p>
<p>So take some time today to learn something new. The way I do this is to select a given problem, and try to solve it with a technology I&rsquo;m not familiar with. For instance &ndash; create a Purchase Order system in Excel, then in Hadoop or MongoDB, or even in flat-files using PowerShell as an interface. No, I&rsquo;m not suggesting any of these architectures are the proper way to solve the PO problem, but taking something concrete that you know well and applying that meta-knowledge to another platform will assist you in exercising the &ldquo;little grey cells&rdquo; and help you and your organization understand what is open to you.</p>
<p>And of course you can do all of this on-premises &ndash; but my recommendation is to check out a cloud platform (my suggestion would of course be Windows Azure :) ) and try it there. Most providers (including Microsoft) provide free time to do that.</p>Creating a Corporate Data Hubhttp://sqlblog.com/blogs/buck_woody/archive/2012/06/26/creating-a-corporate-data-hub.aspxTue, 26 Jun 2012 14:36:41 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:44090BuckWoody<p>The Windows Azure Marketplace has a rich assortment of data and software offerings for you to use – a type of Software as a Service (SaaS) for IT workers, not necessarily for end-users. Among those offerings is the “Data Hub” – a&#160; codename for a project that ironically actually does what the codename says. </p> <p>In many of our organizations, we have multiple data quality issues. Finding data is one problem, but finding it just once is often a bigger problem. Lots of departments and even individuals have stored the same data more than once, and in some cases, made changes to one of the copies. It’s difficult to know which location or version of the data is authoritative.</p> <p>Then there’s the problem of accessing the data. It’s fairly straightforward to publish a database, share or other location internally to store the data. But then you have to figure out who owns it, how it is controlled, and pass out the various connection strings to those who want to use it. And then you need to figure out how to let folks access the internal data externally – bringing up all kinds of security issues. Finally, in many cases our user community wants us to combine data from the internally sources with external data, bringing up the security, strings, and exploration features up all over again.</p> <p>Enter the Data Hub. This is an online offering, where you assign an administrator and data stewards. You import the data into the service, and it’s available to you - and only you and your organization if you wish. </p> <p><img src="http://www.microsoft.com/global/en-us/sqlazurelabs/PublishingImages/datahub-image3-large.jpg" width="447" height="376" /></p> <p>The basic steps for this service are to set up the portal for your company, assign administrators and permissions, and then you assign data areas and import data into them. From there you make them discoverable, and then you have multiple options that you or your users can access that data. You’re then able, if you wish, to combine that data with other data in one location. </p> <p>So how does all that work? What about security? Is it really that easy? And can you really move the data definition off to the Subject Matter Experts (SME’s) that know the particular data stack better than the IT team does?</p> <p>Well, nothing good is easy – but using the Data Hub is actually pretty simple. I’ll give you a link in a moment where you can sign up and try this yourself. Once you sign up, you assign an administrator. From there you’ll create data areas, and then use a simple interface to bring the data in. All of this is done in a portal interface – nothing to install, configure, update or manage. </p> <p>After the data is entered in, and you’ve assigned meta-data to describe it, your users have multiple options to access it. They can simply use the portal – which actually has powerful visualizations you can use on any platform, even mobile phones or tablets. </p> <p><img src="http://i.msdn.microsoft.com/dynimg/IC498608.gif" width="459" height="213" />&#160;</p> <p>&#160;</p> <p>Your users can also hit the data with Excel – which gives them ultimate flexibility for display, all while using an authoritative, single reference for the data. Since the service is online, they can do this wherever they are – given the proper authentication and permissions. You can also hit the service with simple API calls, like this one from C#: <a title="http://msdn.microsoft.com/en-us/library/hh921924" href="http://msdn.microsoft.com/en-us/library/hh921924">http://msdn.microsoft.com/en-us/library/hh921924</a>&#160;</p> <p>You can make HTTP calls instead of code, and the data can even be exposed as an OData Feed. As you can see, there are a lot of options. </p> <p>You can check out the offering here: <a title="http://www.microsoft.com/en-us/sqlazurelabs/labs/data-hub.aspx" href="http://www.microsoft.com/en-us/sqlazurelabs/labs/data-hub.aspx">http://www.microsoft.com/en-us/sqlazurelabs/labs/data-hub.aspx</a> and you can read the documentation here: <a title="http://msdn.microsoft.com/en-us/library/hh921938" href="http://msdn.microsoft.com/en-us/library/hh921938">http://msdn.microsoft.com/en-us/library/hh921938</a></p>BlobShare: Corporate Filesharing with Security and Controlhttp://sqlblog.com/blogs/buck_woody/archive/2012/05/08/blobshare-corporate-filesharing-with-security-and-control.aspxTue, 08 May 2012 12:49:21 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:43261BuckWoody<p>We’ve all done it. We have a file we need to transfer to someone&#160; - but it’s big. Really big. “Big”, in this case, is defined as “bigger than my corporate e-mail will let me send.” So of course we scout around, and find a free file share location. We plop it in, and send a link to the other person. Problem solved!</p> <p>No, problem created. You see, many of these file-sharing sites have some…interesting…language in their terms. Sure, for a picture of aunt Sally riding that llama on the vacation, who cares if someone else steals it and puts it on an advertisement for a <a href="http://en.wikipedia.org/wiki/Elbonia#Elbonia" target="_blank">shipping company from Elbonia</a>? But most of these offerings were not designed for a corporation with private or secure information - sometimes your private or secure information. </p> <p>So what’s a company to do? You have to let people send files - they are just going to work around IT if you don’t - but you need a way to keep it secure, and maintain a chain of custody in some cases. There is a solution.</p> <p><a href="http://blobshare.codeplex.com/" target="_blank">There’s a free Codeplex project called “BlobShare” that we have for you here</a>. You pull it down, follow the instructions, and deploy it to Windows and SQL Azure. From there, you have a clean, safe, secure, controlled interface to let your folks share files. </p> <p><img src="http://blogs.msdn.com/cfs-filesystemfile.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-26-94-metablogapi/7870.image_5F00_3558ACBA.png" /></p> <p>There’s a full post on the inner-workings and the design of this code - and it’s just code. You can change it, use your own logo, restrict it further, loosen it up, whatever you like. That’s the point of Platform as a Service - control and ease. </p> <p><a href="http://blogs.msdn.com/b/vbertocci/archive/2011/10/31/blobshare-sample-acs-protected-file-sharing.aspx">http://blogs.msdn.com/b/vbertocci/archive/2011/10/31/blobshare-sample-acs-protected-file-sharing.aspx</a></p>Windows Azure Use Case: Supplementing Infrastructurehttp://sqlblog.com/blogs/buck_woody/archive/2011/11/01/windows-azure-use-case-supplementing-infrastructure.aspxTue, 01 Nov 2011 13:22:01 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:39516BuckWoody<p>I’ve explained before that Windows Azure is a Platform as a Service - at its simplest, that means that you write software and Azure runs it for you. But what if you are a shop that normally buys “off the shelf” software, and the only software you write is an internal utility here and there - can you still use Windows Azure?</p> <p>Absolutely. Windows Azure is made up of several components, such as computing, storage and other objects you can call in code. And as such, some companies have extended software and hardware you can use that is backed by Windows Azure. In some cases that’s simply software you can use like you would any other - making Azure a “Software as a Service”, which I’ll write about later.</p> <p>But in other cases the software a vendor provides a utility you can use for your infrastructure. For instance - “BlobShare”, a free codeplex offering, allows you to create a place where you can upload and download files securely. It allows you to hook in your local Windows accounts (without sending names and passwords over the Internet) or even Google, LiveID or Yahoo logins to access the files. You can think of it as a “corporate DropBox”. Download it and read more about it here: <a href="http://blobshare.codeplex.com/">http://blobshare.codeplex.com/</a> and learn how to hook in Active Directory here: <span style="font-family:'Calibri','sans-serif';font-size:11pt;mso-fareast-font-family:'Times New Roman';mso-bidi-font-family:'Times New Roman';mso-ansi-language:en-us;mso-fareast-language:en-us;mso-bidi-language:ar-sa;"><a href="http://blogs.msdn.com/b/vbertocci/archive/2011/10/31/blobshare-sample-acs-protected-file-sharing.aspx"><u><font color="#0000ff" size="2" face="Arial">http://blogs.msdn.com/b/vbertocci/archive/2011/10/31/blobshare-sample-acs-protected-file-sharing.aspx</font></u></a></span></p> <p><span style="font-family:'Calibri','sans-serif';font-size:11pt;mso-fareast-font-family:'Times New Roman';mso-bidi-font-family:'Times New Roman';mso-ansi-language:en-us;mso-fareast-language:en-us;mso-bidi-language:ar-sa;"><img src="http://download.codeplex.com/Download?ProjectName=blobshare&amp;DownloadId=295232" width="634" height="306" /></span></p> <p>Another option along these lines is using a hardware appliance to store data in Azure. This has a particular attraction, since some of these appliances (<a href="http://www.storsimple.com/product-overview/" target="_blank">such as the offerings from StorSimple</a>) do more than just act as a storage target. The device plugs into the network as storage, moves the data to Windows Azure, de-duplicates (thus saving storage costs), encrypts, acts as a backup device, works with SharePoint and more. No coding is required to use their solution, and it can even act as a Disaster Recovery site. </p> <p>Storage isn’t the only way to use Windows Azure in place of infrastructure. Using the Application Fabric Service Bus, you can perform some functions of data access and transfer between companies that used to take a VPN setup - which is no longer required. the caching function in the Service Bus can even relieve the need to upgrade a server for performance. And of course SQL Azure allows you to access a SQL Server database without any server at all. </p> <p>So if you’re thinking that Windows and SQL Azure aren’t for you, think again. Start with the problem you have, and see what options you have for solving it. </p>Big Data and the Cloud - More Hype or a Real Workload?http://sqlblog.com/blogs/buck_woody/archive/2011/10/18/big-data-and-the-cloud-more-hype-or-a-real-workload.aspxTue, 18 Oct 2011 13:57:36 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:39156BuckWoody<p>Last week Microsoft announced several new offerings for “Big Data” - and since I’m a stickler for definitions, I wanted to make sure I understood what that really means. What is “Big Data”? What size hard drive is that? After all, my laptop has 1TB of storage - is my laptop “Big Data”?</p> <p>There are actually a few definitions for this term, most notably those involving the <a href="http://nosql.mypopescu.com/post/9621746531/a-definition-of-big-data" target="_blank">“Four V’s” Volume, Velocity, Variety and Variability</a>. Others <a href="http://nosql.mypopescu.com/post/10120087314/big-data-and-the-4-vs-volume-velocity-variety" target="_blank">disagree with this</a> definition. I tend to try and get things into their simplest form, so I’m using this definition for myself:</p> <p align="center"><font color="#c0504d" size="3">Big data is defined as a <em>large set </em>of <em>computationally expensive </em>data that is <em>worked on simultaneously</em>.</font> </p> <p>Let me flesh that out a&#160; little. To be sure, “Big Data” has a larger size than say a few megabytes. The reason this is important is that it takes special hardware to be able to move large sets of data around, store it, process it and so on. (<font color="#c0504d">large set</font>)</p> <p>If you store a LOT of data, but only use a small portion of it at a time, that really isn’t super-hard to do. It’s mainly a storage issue at that point. But, if you do need to work with a large portion of the data at one time, then the memory, CPU and transfer components of the system have to adapt to be responsive - new ways to work with that data (game theory, knot-algorithms, map-reduce, etc.) need to be brought into play. (<font color="#c0504d">computationally expensive</font>)</p> <p>Once that data is loaded into the processing area (memory or whatever other mechanism is used) it must be worked on in parallel to come back in a reasonable time. You have two options here - you can scale the system up with more internal hardware (CPU’s, memory and so on) or you can scale it out to have multiple systems work on it at the same time using paradigms such as map/reduce and so on. Actually, when you lay this out in an architecture diagram, scale up or out doesn’t actually change the logical structure of the process - in scale out the network becomes the bus, and the nodes become more RAM and computing power. Of course, there are changes in code for how you stitch the workload back together. (<font color="#c0504d">worked on simultaneously</font>)</p> <p>So back to the original question. Is Big Data, as I have defined it here, a workload for Windows and SQL Azure? Absolutely! In fact, it’s probably one of the main workloads, and I believe it represents the latest, and perhaps also the earliest frontier of computing. Jim <a href="http://research.microsoft.com/en-us/um/people/gray/" target="_blank">Gray, a former researcher here at Microsoft and a hero of mine, was working on this very topic.</a> I believe as he did - all computing is simply an interface over data. </p> <p>Microsoft has multiple offerings on the topic of Big Data. In posts that follow from myself and my co-workers, we’ll explore when and where you use each one. Whether you are a data professional or a developer, this is the new frontier - <a href="http://www.straightpathsql.com/archives/2011/10/microsoft-loves-your-big-data/" target="_blank">don’t wait to educate yourself</a> on how to leverage Big Data for your organization. </p> <p><strong>Hadoop on Windows Azure and SQL Server&#160; </strong>- Microsoft’s <a href="http://www.hortonworks.com/the-whys-behind-the-microsoft-and-hortonworks-partnership/" target="_blank">partnership to include Hadoop workloads on Windows Azure</a> and <a href="http://www.microsoft.com/download/en/details.aspx?id=27584" target="_blank">SQL Server/Parallel Data Warehouse (PDW)</a></p> <p><strong>LINQ to HPC </strong>- Microsoft’s High-Performance Computing SKU of <a href="http://blogs.technet.com/b/windowshpc/archive/2011/05/20/dryad-becomes-linq-to-hpc.aspx" target="_blank">HPC is now in Azure</a></p> <p><strong>Windows Azure Table Storage </strong>- A <a href="http://msdn.microsoft.com/en-us/library/windowsazure/hh508997.aspx" target="_blank">key/value pair type storage with full partitioning</a> that is immediately consistent, able to handle huge loads of data and works with any REST-compatible language</p> <p>&#160;<strong>Other offerings </strong>- Including the new <a href="http://www.microsoft.com/en-us/sqlazurelabs/default.aspx" target="_blank">Data Explorer</a>, <a href="http://research.microsoft.com/en-us/news/headlines/daytona-071811.aspx" target="_blank">Project Daytona (with a Big Data Toolkit for Scientists and researchers)</a>, <a href="http://www.microsoft.com/sqlserver/en/us/future-editions/SQL-Server-2012-breakthrough-insight.aspx" target="_blank">Power View</a> and more. </p> <p>The era of Big Data is here. And you can use Windows and SQL Azure to bring it to your organization. </p>Creating a Distributed Computing System Using a Windows Azure Queuehttp://sqlblog.com/blogs/buck_woody/archive/2011/10/11/creating-a-distributed-computing-system-using-a-windows-azure-queue.aspxTue, 11 Oct 2011 13:12:42 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:38990BuckWoody<p style="margin:0in 0in 0pt;" class="MsoNormal"><font size="3"><font color="#000000"><font face="Calibri">The Windows Azure Queue component, like all Windows Azure components (Roles, Storage, App Fabric, SQL Azure) can be used by itself or with other Windows Azure components. That’s why I refer to Windows Azure as “Distributed Computing” rather than “cloud”. </font></font></font></p> <p style="margin:0in 0in 0pt;" class="MsoNormal"><font color="#000000" size="3" face="Calibri"></font></p> <p style="margin:0in 0in 0pt;" class="MsoNormal"><font size="3"><font color="#000000"><font face="Calibri">Having a distributed off premise queue has a lot of use-cases. An interesting use-case is a company that wanted to harness the power of all of the PC’s and laptops in the company when they were not being used throughout the day. A developer wrote a screen-saver program that connected to an Azure Queue, pulling work off of the queue and placing an entry when it was done. In essence he had a partially connected distributed work relay system, and since he used a Windows Azure Queue, the system worked from anywhere in the world. </font></font></font></p> <p style="margin:0in 0in 0pt;" class="MsoNormal"> <p><font color="#000000" size="3" face="Calibri">&#160;</font></p> </p> <p style="margin:0in 0in 0pt;" class="MsoNormal"><font size="3"><font color="#000000"><font face="Calibri">He uses an on-site central server (which was actually only a workstation-level system) that holds the computations in a scatter/gather paradigm. The computations are broken into less-than-8K chunks, so that it fits within a message. The server connects to a Windows Azure Queue, and places the message marked for computation. It also scrubs the Queue for completed work, and as part of the process puts that kind of message into a mapping function (queues are not guaranteed a message order). </font></font></font></p> <p style="margin:0in 0in 0pt;" class="MsoNormal"> <p><font color="#000000" size="3" face="Calibri">&#160;</font></p> </p> <p style="margin:0in 0in 0pt;" class="MsoNormal"><font size="3"><font color="#000000"><font face="Calibri">The workstations that are not being used (even those systems at remote workers and travelers) connect to the same Windows Azure Queue when the system is not being used for a period of time, when the screen saver kicks in. It then takes one message from the queue, computes the information, and then sets a new message for the server to pick up with the answer. The workstation then deletes the message. </font></font></font></p> <p style="margin:0in 0in 0pt;" class="MsoNormal"> <p><font color="#000000" size="3" face="Calibri">&#160;</font></p> </p> <p style="margin:0in 0in 0pt;" class="MsoNormal"><font size="3"><font color="#000000"><font face="Calibri">The Server picks up the completed work, processes it and then deletes that queue message. He also added logic to process messages for computation on the server as well, when the server function of adding work is not required. </font></font></font></p> <p style="margin:0in 0in 0pt;" class="MsoNormal"> <p><font color="#000000" size="3" face="Calibri">&#160;</font></p> </p> <p style="margin:0in 0in 0pt;" class="MsoNormal"> <p><font color="#000000" size="3" face="Calibri"><a href="http://blogs.msdn.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-79-79-metablogapi/2438.AzureQueueDistributedSystem_5F00_2.png"><img style="background-image:none;border-bottom:0px;border-left:0px;padding-left:0px;padding-right:0px;display:inline;border-top:0px;border-right:0px;padding-top:0px;" title="AzureQueueDistributedSystem" border="0" alt="AzureQueueDistributedSystem" src="http://blogs.msdn.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-79-79-metablogapi/3603.AzureQueueDistributedSystem_5F00_thumb.png" width="708" height="919" /></a></font></p> </p> <p style="margin:0in 0in 0pt;" class="MsoNormal"> <p>&#160;</p> </p> <p style="margin:0in 0in 0pt;" class="MsoNormal"><font size="3"><font color="#000000"><font face="Calibri">There are a few caveats here. This works because of the mapping function on the head server. Order is not guaranteed, so he includes a number for the function step as part of the message body, which cuts the size a bit. Also, he’s careful to watch the encoding, since Azure will hand binary back in Base64 format. </font></font></font></p> <p style="margin:0in 0in 0pt;" class="MsoNormal"> <p><font color="#000000" size="3" face="Calibri">&#160;</font></p> </p> <p style="margin:0in 0in 0pt;" class="MsoNormal"><font size="3"><font color="#000000"><font face="Calibri">He’s found that there are enough systems to ensure that the messages are cleared every few days – important, since the Windows Azure Queue ages out after seven days. Also, he’s careful to use the CloudQueue.PeekMessage function when he wants to monitor the system – that function ensures that the message status doesn’t reset as “read” when he accesses it. </font></font></font></p> <p style="margin:0in 0in 0pt;" class="MsoNormal"> <p><font color="#000000" size="3" face="Calibri">&#160;</font></p> </p> <p style="margin:0in 0in 0pt;" class="MsoNormal"><font size="3"><font color="#000000"><font face="Calibri">This is a great example of using the “cloud” as what it is intended to be – a distributed architecture you can use as needed to solve a business problem. It’s not an “all or nothing” proposition, but instead it is simply another set of components to use where you need them. </font></font></font></p>Rip and Replace or Extend and Embrace?http://sqlblog.com/blogs/buck_woody/archive/2011/09/13/rip-and-replace-or-extend-and-embrace.aspxTue, 13 Sep 2011 11:20:05 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:38437BuckWoody<p>As most of you know, I don&rsquo;t like the term &ldquo;cloud&rdquo; very<br />much. It isn&rsquo;t defined, which means it can be anything. I prefer &ldquo;distributed<br />computing&rdquo;, which is more technically accurate and describes what you&rsquo;re doing<br />in more concrete terms.</p>
<p>So when you think about Windows and SQL Azure, you don&rsquo;t<br />have to think about an entire product &ndash; you can use parts of the system<br />together or independently to accomplish what you need to do. You can use the<br />computing functions, storage, and more and more I see folks leverage the<br />Service Bus to enable current applications to expose things to the web.</p>
<p>And that brings up the point of this post. Once you decide<br />that a distributed architecture works to solve a problem, you&rsquo;re faced with a<br />decision: should you completely re-write your architecture to take advantage of<br />the current systems or should you just fold in new code that makes the data or<br />function available to the web?</p>
<p>Of course, the answer is always &ldquo;it depends&rdquo; on the situation<br />&ndash; and it does. But unless you&rsquo;re fixing a problem with current code, I usually<br />advocate a migration approach. That means at the very least retaining the<br />business logic (again, unless it&rsquo;s not currently working) and as much of the<br />code as you can. In fact, if you follow this paradigm, you&rsquo;re on your way to<br />making a Service Bus out of the functions you currently have. You can expose<br />the results of a system rather than opening the system up. Let&rsquo;s take an<br />example.</p>
<p>Assume for a moment that you have an order-taking system<br />on-premise. That system performs many functions, one of which might creating a<br />Purchase Order. Your system might be enclosed, meaning that it has an<br />application that talks to a middle-tier, and then from there to a database<br />system. A query is generated from a screen, and passed along to eventually<br />compute, store and return a Purchase Order Number, along with other<br />information. Imagine now that you wire up the code not only to return the PO<br />number to the client, but to make that number available on an endpoint &ndash;<br />actually really not that hard to do.</p>
<p>Now you can make that PO number available to the web using<br />Azure. You could restrict who can make that call to the system, or open it up<br />to a broader audience. Or instead of the PO Number, you could make a product<br />list available. And you can go further than that &ndash; EBay, for instance, uses the<br />OData protocol (which is very cool in and of itself) which you can query from<br />the web. You could compare your company&rsquo;s product catalog to what is on EBay,<br />and list the items you have there if there are no competitors in that space.<br />And on and on it goes.</p>
<p>So the point is this &ndash; where you can, retain what works.<br />Fold in systems like Azure where they make sense. Extend and Embrace.</p>The Conundrum of Cloud Computing Evidencehttp://sqlblog.com/blogs/buck_woody/archive/2011/07/26/the-conundrum-of-cloud-computing-evidence.aspxTue, 26 Jul 2011 12:56:08 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:37317BuckWoody<p>“Who is using Windows Azure? How are they doing it?” </p> <p>I get asked this all the time as I speak to our clients when we mention Windows or SQL Azure as a possible solutions to an architectural problem the company has. I completely understand the question. I’ve worked far longer outside of Microsoft than here, and one of my roles as a Systems Architect was to select solutions from a range of possibilities. When you’re faced with a decision like that, it saves a ton of time if you can find out where others have done the same thing and what problems and successes they had with this or that approach. I’m not a marketing person, so I work with clients directly all the time, and so folks know I’ve seen the way others do things. And they want to know what someone else has done before they try it. </p> <p>But therein lies the rub.</p> <p>Most folks aren’t willing to talk about their internal infrastructure much. <a href="http://technet.microsoft.com/en-us/library/bb687780.aspx" target="_blank">We talk about how we do things here at Microsoft quite frequently, and if you’re not using this resource, definitely go research what we’ve done</a>. In fact, I don’t think I’ve seen many other technology companies with this depth of information. For instance, here’s a quick list of the areas we talk about on our internal systems: </p> <li><a id="ctl00_MTCS_main_ctl32" href="http://technet.microsoft.com/en-us/library/bb687768.aspx"><u><font color="#0066cc">BizTalk</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl33" href="http://technet.microsoft.com/en-us/library/bb687782.aspx"><u><font color="#0066cc">Exchange Server</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl34" href="http://technet.microsoft.com/en-us/library/ee862420.aspx"><u><font color="#0066cc">Internet Explorer</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl35" href="http://technet.microsoft.com/en-us/library/gg605859.aspx"><u><font color="#0066cc">Lync</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl36" href="http://technet.microsoft.com/en-us/library/cc982289.aspx"><u><font color="#0066cc">Microsoft Desktop Optimization Pack</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl37" href="http://technet.microsoft.com/en-us/library/dd197309.aspx"><u><font color="#0066cc">Microsoft Dynamics</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl38" href="http://technet.microsoft.com/en-us/library/dd197310.aspx"><u><font color="#0066cc">Microsoft Forefront</font></u></a></li> <li><strong>Office System</strong> <ul> <li><a id="ctl00_MTCS_main_ctl39" href="http://technet.microsoft.com/en-us/library/bb687788.aspx"><u><font color="#0066cc">Office Communications Server</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl40" href="http://technet.microsoft.com/en-us/library/bb687797.aspx"><u><font color="#0066cc">SharePoint Server</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl41" href="http://technet.microsoft.com/en-us/library/bb687792.aspx"><u><font color="#0066cc">Office</font></u></a></li></ul></li> <li><a id="ctl00_MTCS_main_ctl42" href="http://technet.microsoft.com/en-us/library/bb687798.aspx"><u><font color="#0066cc">SQL Server</font></u></a></li> <li><strong>System Center</strong> <ul> <li><a id="ctl00_MTCS_main_ctl43" href="http://technet.microsoft.com/en-us/library/bb687796.aspx"><u><font color="#0066cc">System Center Configuration Manager</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl44" href="http://technet.microsoft.com/en-us/library/bb736232.aspx"><u><font color="#0066cc">System Center Data Protection Manager</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl45" href="http://technet.microsoft.com/en-us/library/bb687791.aspx"><u><font color="#0066cc">System Center Operations Manager</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl46" href="http://technet.microsoft.com/en-us/library/gg236616.aspx"><u><font color="#0066cc">System Center Service Manager</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl47" href="http://technet.microsoft.com/en-us/library/hh124528.aspx"><u><font color="#0066cc">Virtual Machine Manager</font></u></a></li></ul></li> <li><a id="ctl00_MTCS_main_ctl48" href="http://technet.microsoft.com/en-us/library/bb687801.aspx"><u><font color="#0066cc">Visual Studio</font></u></a></li> <li><strong>Windows</strong> <ul> <li><a id="ctl00_MTCS_main_ctl49" href="http://technet.microsoft.com/en-us/library/gg605186.aspx"><u><font color="#0066cc">Windows Azure</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl50" href="http://technet.microsoft.com/en-us/library/bb687804.aspx"><u><font color="#0066cc">Windows Client</font></u></a></li> <li><a id="ctl00_MTCS_main_ctl51" href="http://technet.microsoft.com/en-us/library/bb687806.aspx"><u><font color="#0066cc">Windows Server</font></u></a></li></ul></li> <li class="content-item"> <p>&nbsp;</p> <li class="content-item"> <p>And yes, Azure as you can see is part of that list - we’ve implemented it everywhere. But when I point out that we’re using IT resources just like any other company, I hear “yeah, but you’re <em>Microsoft</em>. You have lots of people to handle the systems, and you can just have the developer who wrote the software come over and fix it if it breaks.” Well, that’s not the way it works at all. When I got here, I met with some folks from Microsoft IT, and I was absolutely shocked at how few people manage our systems. From Systems Administrators all the way to the DBA’s, the ratio of technical resource to machines is really extraordinary, and I had far more folks managing systems for me in my other companies than we have here. And no, we don’t send developers to fix production systems “just for us”, although we will do that when we are using a beta of something that we haven’t even released to CTP yet. </p> <p>But even with this evidence of ourselves, clients want to “see how others are doing things with Azure”. <a href="http://www.microsoft.com/windowsazure/evidence/" target="_blank">We do have the customer evidence site, and we have lots of case-studies there</a>. But many of these lack technical details, again, because folks aren’t always willing to share that. That’s understandable - for instance, even though Microsoft does it, I’m not sure I would have the Visio diagrams for my company’s applications posted in a public place. There may even be legal or competitive reasons <em>not </em>to do that. So as you peruse that list, keep in mind these are only the companies that are willing to talk in public about their experience - we have far more on Azure than this list. </p> <p><a href="http://blogs.msdn.com/b/jmeier/archive/2011/07/25/microsoft-cloud-case-studies-at-a-glance.aspx" target="_blank">J.D. Meiers has a good blog post entry</a> of a rundown of the companies that have released case studies on Azure. And be assured that <a href="http://msdn.microsoft.com/en-us/library/ff898430.aspx" target="_blank">we have a patterns and practices group that works with the teams here at Azure</a> to make sure we apply what we learn as we help each client. And many of our partners that you use to help you implement technology are here at Microsoft this week, sharing customer stories (with permission, of course), which allows us to learn from them on how you’re doing things - what works and what doesn’t. </p> <p>As time goes on I think you’ll see more patterns emerge from the engagements we’ve done. Right now those are sometimes a competitive advantage, so you’ll see us generalize the patterns into content we can release. For now, check out those case studies, and do make use of the Internal Microsoft IT sites. They really are quite detailed, and have a lot of information you can use. </p>SQL Azure Use Case: Shared Data Hubhttp://sqlblog.com/blogs/buck_woody/archive/2011/04/05/sql-azure-use-case-shared-data-hub.aspxTue, 05 Apr 2011 14:10:50 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:34672BuckWoody<p><span style="font-size:x-small;"><em><span style="font-size:small;">This is one in a series of posts on when and where to use a distributed architecture design in your organization's computing needs. You can find the main post here: </span><a href="http://blogs.msdn.com/b/buckwoody/archive/2011/01/18/windows-azure-and-sql-azure-use-cases.aspx"><span style="font-size:small;"><u><font color="#800080">http://blogs.msdn.com/b/buckwoody/archive/2011/01/18/windows-azure-and-sql-azure-use-cases.aspx</font></u></span></a><span style="font-size:small;"> </span></em></span></p> <p><strong><span style="font-size:small;">Description:</span></strong></p> <p><font size="2">Organizations often need to share all or part of a data set, which is consumed by other systems. These systems can be on-premise or at another location, or even at a different organization. </font></p> <p><font size="2">Many times these systems use a well-defined data interchange system, such as EDI or other standards. In the case of a trusted system, simply using a direct connection into another database is the process used to transfer data. This process might be one-way or bi-directional.</font></p> <p><font size="2">But there are systems that transfer data back and forth in stages using intermediate systems. A typical data flow in this case looks similar to the following:</font></p> <p><font size="2"><a href="http://blogs.msdn.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-79-79-metablogapi/7823.SADH_2D00_1_5F00_2.png"><img style="background-image:none;border-bottom:0px;border-left:0px;padding-left:0px;padding-right:0px;display:inline;border-top:0px;border-right:0px;padding-top:0px;" title="SADH-1" border="0" alt="SADH-1" src="http://blogs.msdn.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-79-79-metablogapi/7206.SADH_2D00_1_5F00_thumb.png" width="550" height="227" /></a></font></p> <p><font size="2">In this example, the owning system contains data set A. This is set to a staging system or server, where the receiving system collects it. The receiving system contains data set B and works with data set A to create a new data set, C. This new data is consumed by the original system to complete the cycle. A concrete example is an inventory control system. Data set A is the original inventory list, shipped to a manufacturer. The manufacturer consumes the inventory available, orders and components, and returns the ordering bid with any changes to the staging server as data set C. The data is consumed by the originating system and components are noted in the overall flow of data set A.</font></p> <blockquote> <p><font size="2"><em>Note: Normally this is solved with a full EDI implementation, but this process is still a common practice.</em></font></p> </blockquote> <p><font size="2">There are other examples, but the general concept is one where the need is for two, possibly untrusted systems to share a common source of data.</font></p> <p><strong><span style="font-size:small;">Implementation:</span></strong> </p> <p><font size="2">One possible solution is to segregate the data that is being transferred into an agree-upon set of entities that can be added or edited real-time, where both systems (or many) feed from the same data set instead of shipping the data. This removes latency, improves data quality, and shares the cost of the data. Also, security is increased because there are no shared logins - each firm gets its own.</font></p> <p><font size="2"><a href="http://blogs.msdn.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-79-79-metablogapi/6232.SADH_2D00_2_5F00_2.png"><img style="background-image:none;border-bottom:0px;border-left:0px;padding-left:0px;padding-right:0px;display:inline;border-top:0px;border-right:0px;padding-top:0px;" title="SADH-2" border="0" alt="SADH-2" src="http://blogs.msdn.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-79-79-metablogapi/0044.SADH_2D00_2_5F00_thumb.png" width="412" height="283" /></a></font></p> <p><font size="2">One consideration with this layout is that the source systems must be altered to use a shared data set. If this is not possible, this is still a possibility as the system can be used as it was before - a data transfer - but the data can be cleansed real-time by both systems. It’s also a more secure and shared-cost system even if used in the original manner.</font></p> <p><font size="2"><strong><span style="font-size:small;">Resources:</span></strong></font> <p><font size="2">Security is a concern in this arrangement, so it’s best to understand exactly how the security works in SQL Azure: <a href="http://msdn.microsoft.com/en-us/library/ff394108.aspx">http://msdn.microsoft.com/en-us/library/ff394108.aspx</a>&#160;</font></p> <p>Another possibility to solve this pattern is to use Data Sync, in many different arrangements that involve SQL Azure. You can learn more about it here: <a href="http://blogs.msdn.com/b/sync/archive/2010/10/07/windows-azure-sync-service-demo-available-for-download.aspx">http://blogs.msdn.com/b/sync/archive/2010/10/07/windows-azure-sync-service-demo-available-for-download.aspx</a></p></p>SQL Azure Use Case: Department Application Data Storehttp://sqlblog.com/blogs/buck_woody/archive/2011/03/22/sql-azure-use-case-department-application-data-store.aspxTue, 22 Mar 2011 13:17:51 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:34310BuckWoody<p><span style="font-size:x-small;"><em><span style="font-size:small;">This is one in a series of posts on when and where to use a distributed architecture design in your organization's computing needs. You can find the main post here: </span><a href="http://blogs.msdn.com/b/buckwoody/archive/2011/01/18/windows-azure-and-sql-azure-use-cases.aspx"><span style="font-size:small;"><u><font color="#800080">http://blogs.msdn.com/b/buckwoody/archive/2011/01/18/windows-azure-and-sql-azure-use-cases.aspx</font></u></span></a><span style="font-size:small;">&#160;</span></em></span></p> <p><strong><span style="font-size:small;">Description:</span></strong></p> <p><font size="2">Most organizations use a single set of enterprise applications where employees do their work. This system is designed to be secure, safe, and to perform well. It is attended and maintained by a staff of trained technical professionals. But not all business-critical work is contained in these systems. Users who have installed software such as Microsoft Office often set up small data-stores and even programs to do their work. This is often because the system is “bounded” - the data or program only affects an individual’s work. </font></p> <p><font size="2">The issue arises when an individual’s data system is shared with other users. In effect the user’s system becomes a server.</font></p> <p><font size="2"><a href="http://blogs.msdn.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-79-79-metablogapi/7142.SQLA_2D00_1_5F00_2.png"><img style="background-image:none;border-bottom:0px;border-left:0px;padding-left:0px;padding-right:0px;display:inline;border-top:0px;border-right:0px;padding-top:0px;" title="SQLA-1" border="0" alt="SQLA-1" src="http://blogs.msdn.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-79-79-metablogapi/7230.SQLA_2D00_1_5F00_thumb.png" width="456" height="154" /></a></font></p> <p><font size="2">There are many issues with this arrangement. The “server” in this case is not secure, safe and does not perform well at higher loads. It may have design issues, and is normally cannot be accessed from outside the office. Users might even install insecure software to allow them to access the system when they are on the road, adding to the insecurity. When the system becomes unavailable or has a design issue, IT may finally become aware of the application’s existence, and it’s criticality. </font></p> <p><a href="http://blogs.msdn.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-79-79-metablogapi/5367.image_5F00_4.png"><font size="2"></font></a></p> <p><strong><span style="font-size:small;">Implementation:</span></strong> </p> <p><font size="2">There are various options to correct this situation. The first and most often used is to bring the system under IT control, re-write or adapt the application to a better design, and run it from a proper server that is backed up, performs well, and is safe and secure. Depending on how critical the application is, how many users are accessing it, and the level of effort, this is often the best choice.</font></p> <p><font size="2">But this means that IT will have to assign resources, including developer and admin time,licenses, hardware and other infrastructure and development resources, and this takes time and adds to budget.</font></p> <p><font size="2">If the application simply needs to have a central data store, then SQL Azure is a possible architecture solution. In smaller applications such as Microsoft Access, data can be separated from the application - in other words, the users can still create reports, forms, queries and so on and yet use SQL Azure as the location of the data store. A minimal investment of IT time in designing the tables and indexes properly then allows the users to maintain their own application. No servers, installations, licenses or maintenance is needed. </font></p> <p><font size="2"><a href="http://blogs.msdn.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-79-79-metablogapi/3513.SQLA_2D00_2_5F00_2.png"><img style="background-image:none;border-bottom:0px;border-left:0px;padding-left:0px;padding-right:0px;display:inline;border-top:0px;border-right:0px;padding-top:0px;" title="SQLA-2" border="0" alt="SQLA-2" src="http://blogs.msdn.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-79-79-metablogapi/1263.SQLA_2D00_2_5F00_thumb.png" width="541" height="298" /></a></font></p> <p><font size="2">Another advantage to this arrangement is that the user’s department can set up the subscription to SQL Azure, which shifts the cost burden to the department that uses the data, rather than to IT. </font></p> <p><font size="2">Microsoft Access is only one example - many programs can use an ODBC connection as a source, and since SQL Azure uses this as a connection method, to the application it appears as any other database. The application then can be used in any location, not just from within the building, so there’s no need for a VPN into the building - the front end application simply goes with the user.</font></p> <p><font size="2">Another option for a front-end is to have the power-user create a LightSwitch application. More of a pure development environment than Microsoft Access, it still offers an easy process for a technical business user to create applications. Pairing this environment with SQL Azure makes for a mobile application that is safe and secure, paid for by the department. Interestingly, if designed properly, the data store can serve multiple kinds of front-end programs simultaneously, allowing a great deal of flexibility and the ability to transfer the data to an enterprise system at a later date if that is needed.</font></p> <font size="2"> <p><strong><span style="font-size:small;">Resources:</span></strong></p> <p><font size="2">Example process of moving a Microsoft Access data back-end to SQL Azure: <a href="http://www.informit.com/guides/content.aspx?g=sqlserver&amp;seqNum=375">http://www.informit.com/guides/content.aspx?g=sqlserver&amp;seqNum=375</a></font></p> <p>More information on LightSwitch: <a href="http://www.microsoft.com/visualstudio/en-us/lightswitch">http://www.microsoft.com/visualstudio/en-us/lightswitch</a>&#160;</p> </font>