Search results matching tags 'PASS' and 'SSIS'http://sqlblog.com/search/SearchResults.aspx?o=DateDescending&tag=PASS,SSIS&orTags=0Search results matching tags 'PASS' and 'SSIS'en-USCommunityServer 2.1 SP2 (Build: 61129.1)SQL Saturday 277–Richmond Precons Are 21 Mar 2014!http://sqlblog.com/blogs/andy_leonard/archive/2014/02/05/sql-saturday-277-richmond-precons-are-21-mar-2014.aspxWed, 05 Feb 2014 18:25:49 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:52735andyleonard<p><a href="http://sqlsaturday.com/277/eventhome.aspx" target="_blank"><img border="0" src="http://sqldatapartners.files.wordpress.com/2013/03/rva.png" /></a></p> <p><a href="http://sqlsaturday.com/277/eventhome.aspx" target="_blank">SQL Saturday #277</a> is going to be held 22 Mar 2014 in Richmond Virginia! </p> <p>There will be three day-long pre-conference sessions on Friday 21 Mar:</p> <ul> <li><a href="http://richmondsql.org/cs2007/blogs/events/archive/2014/01/03/sqlsaturday-pre-con-with-robert-davis.aspx" target="_blank">Performance Tuning and Troubleshooting Like a Pro</a> by Robert L. Davis (<a href="http://twitter.com/sqlsoldier" target="_blank">@sqlsoldier</a>)</li> <li><a href="http://richmondsql.org/cs2007/blogs/events/archive/2014/01/04/sqlsaturday-pre-con-with-wayne-sheffield-and-jason-brimhall.aspx" target="_blank">Murder They Wrote</a> by Wayne Sheffield (<a href="https://twitter.com/DBAWayne/" target="_blank">@DBAWayne</a>) and Jason Brimhall (<a href="http://twitter.com/sqlrnnr" target="_blank">@sqlrnnr</a>)</li> <li><a href="http://richmondsql.org/cs2007/blogs/events/archive/2014/01/04/sqlsaturday-pre-con-with-andy-leonard.aspx" target="_blank">SSIS Design Patterns</a> by me (<a href="https://twitter.com/AndyLeonard" target="_blank">@AndyLeonard</a>)</li> </ul> <p>You can find details of the precons <a href="http://sqlsaturday.com/277/eventhome.aspx" target="_blank">here</a> and you can register <a href="https://www.raceit.com/Register/?event=24549" target="_blank">here</a>!</p> <p>I hope to see you there.</p> <p>:{&gt;</p>Automate Your ETL Infrastructure with SSIS and PowerShellhttp://sqlblog.com/blogs/sergio_govoni/archive/2013/12/25/automate-your-etl-infrastructure-with-ssis-and-powershell.aspxWed, 25 Dec 2013 21:57:00 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:52244Sergio Govoni<p>SQLPASS TV channel has been published the session <a href="http://www.youtube.com/watch?v=rlrw0dNmFdU&amp;list=PLoGAcXKPcRvbTr23ujEN953pLP_nDyZJC" target="_blank" mce_href="http://www.youtube.com/watch?v=rlrw0dNmFdU&amp;list=PLoGAcXKPcRvbTr23ujEN953pLP_nDyZJC">Automate Your ETL Infrastructure with SSIS and PowerShell</a> that Allen White (<a href="https://twitter.com/SQLRunr" target="_blank" mce_href="https://twitter.com/SQLRunr">@SQLRunr</a>) has done during the PASS Summit 2013.&nbsp;</p>
<p><b>Abstract</b></p>
<p>Much of your ETL process flow consists of packages that are very similar in structure, capturing data from a single source and transferring that to a single destination. Creating the individual packages can be tedious, and it’s easy to miss something in the process of generating the same basic package over and over again. BI Markup Language makes it easy to build new packages, and PowerShell makes creating the BIML scripts easy. In this session, we’ll see how you can use PowerShell to generate dozens of SSIS packages doing similar tasks from a defined set of ETL sources.</p>
<p><b>Link</b></p>
<p><a href="http://www.youtube.com/watch?v=rlrw0dNmFdU&amp;list=PLoGAcXKPcRvbTr23ujEN953pLP_nDyZJC">http://www.youtube.com/watch?v=rlrw0dNmFdU&amp;list=PLoGAcXKPcRvbTr23ujEN953pLP_nDyZJC</a>&nbsp;</p><p>Enjoy the session and thanks to Allen White!</p>PASS DW/BI Virtual Chapter Upcoming Sessions (December 2013)http://sqlblog.com/blogs/sergio_govoni/archive/2013/12/06/pass-dw-bi-virtual-chapter-upcoming-sessions-december-2013.aspxFri, 06 Dec 2013 23:33:00 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:52043Sergio Govoni<p><span class="hps">Let me point out the upcoming live events scheduled for December 2013 organized by&nbsp;<a href="http://bi.sqlpass.org/" target="_blank" mce_href="http://bi.sqlpass.org/">PASS Business Intelligence Virtual Chapter</a>.</span></p><p>&nbsp;</p><p><b>Create and Load a Staging Environment from Scratch in an Hour with Biml</b></p><p>Date: Thursday 12 December Noon PST / 3 PM EST / 8 PM GMT<br>Speaker: Scott Currie<br>URL: <a href="https://attendee.gotowebinar.com/register/7424713205660411905" target="_blank" mce_href="https://attendee.gotowebinar.com/register/7424713205660411905">https://attendee.gotowebinar.com/register/7424713205660411905</a> </p><p>Business Intelligence Markup Language (Biml) automates your BI patterns and eliminates the manual repetition that consumes most of your SSIS development time. During this hour long presentation, Scott Currie from Varigence will use the free BIDSHelper add-in for BIDS and SSDT to introduce Biml and use to automatically generate large quantities of custom SSIS packages. The session will be largely demonstration driven, and reusable sample code will be distributed for you to use in your own projects. Using a live-typing approach, Scott will start from scratch and by the end of the session create a full-blown staging environment. This will include the creation of *hundreds* of target table creation scripts, data load packages, data scrubbing rules, logging, and more. The best part is that you can freely reuse the code in your own environment just by changing the connection strings - or make small changes to implement your own data load patterns.</p><p>&nbsp;</p><p><b>Inferred Dimension Members within MDS and SSIS</b></p><p>Date: Monday 16 December 3 PM PST / 6 PM EST / 11 PM GMT<br>Speaker: Reza Rad<br>URL: <a href="https://attendee.gotowebinar.com/register/7123625140094491905" target="_blank" mce_href="https://attendee.gotowebinar.com/register/7123625140094491905">https://attendee.gotowebinar.com/register/7123625140094491905</a> </p><p>Combining Master Data Services with Data Warehouses, will cause some challenges in ETL Scenarios. In this session we will go through a demo of Inferred Dimension Members implementation with SSIS considering the fact that MDS keeps the single version of truth for the dimension record. In this session you will learn how we will write back new record's data into MDS entity as an Inferred member. The staging structure of Master Data Services and Batch Processing will be used for this. Then you will learn what is the best practice to add the inferred record into Data Warehouse dimension. Updating the existing dimension member also would consider the Inferred member and apply SCD types only if this is not an inferred Member.</p><p>&nbsp;</p><p><b>Guerrilla MDS/MDM The Road To Data Governance</b></p><p>Date: Thursday 19 December Noon PST / 3 PM EST / 8 PM GMT<br>Speakers: Ira Whiteside and Victoria Stasiewicz<br>URL: <a href="https://attendee.gotowebinar.com/register/336644427020709122" target="_blank" mce_href="https://attendee.gotowebinar.com/register/336644427020709122">https://attendee.gotowebinar.com/register/336644427020709122</a> </p><p>Ira and Vic's session "Guerrilla MDS" will be a walk-through of a real-world implementation for a master data model (MDM) and metadata mart utilizing SSIS, MDS and POWER BI EXCEL add-ins as well as applying proper data quality techniques. We will walk through in detail the processes necessary for utilizing the complete MDS functionality as follows: creating entities attribute, relating entities the domain based attributes, staging leave table, updated entity content, apply business rules, create subscription view and set up security. Source code for all samples and PowerPoint will be made available.&nbsp;</p>24HOP: BI Workload Follow-Uphttp://sqlblog.com/blogs/stacia_misner/archive/2011/09/08/38352.aspxThu, 08 Sep 2011 16:16:19 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:38352smisner<p>Yesterday, Denny Cherry (<a href="http://www.mrdenny.com" target="_blank">blog</a>|<a href="http://www.twitter.com/mrdenny" target="_blank">twitter</a>) and I co-presented a <a href="http://www.sqlpass.org/24hours/fall2011/default.aspx" target="_blank">24HOP session for the Fall 2011 lineup</a>, “So How Does the BI Workload Impact the Database Engine?” 24HOP stands for 24 Hours of PASS and is a semiannual roundup of speakers from the SQL Server community. Initially, this event consisted of 24 consecutive sessions, each lasting an hour, but later it became a two-day event with 12 consecutive sessions each day. The sessions are free to attend and feature many great topics covering the spectrum of SQL Server things to know. Even if you missed previous 24HOP events, you can always go back and view recordings of sessions that interest you at the 24HOP site for <a href="http://www.sqlpass.org/24hours/spring2011/" target="_blank">Spring 2011</a> and <a href="http://www.sqlpass.org/24hours/fall2010/" target="_blank">Fall 2010</a>.</p> <p>And if you missed Denny and me yesterday, a recording will be available in a couple of weeks and I’ll update this post with a link. Our hour-long session for 24HOP was a sneak preview of our upcoming half-day session of the same name that we’ll be presenting at the <a href="http://www.sqlpass.org/summit/2011/" target="_blank">PASS Summit</a> in Seattle on Thursday, October 13, 2011 from 1:30 pm to 4:30 PM. In our <a href="http://www.sqlpass.org/summit/2011/Speakers/CallForSpeakers/SessionDetail.aspx?sid=1134" target="_blank">half-day session</a>, we’ll dig into the details and spend more time on database engine analysis, whereas in our 24HOP session, we focused on reviewing the architecture and highlighting the connection between BI components and the database engine.</p> <p>We were able to answer a few questions at the end, but one question in particular could not be answered easily in the time allotted in a single sentence or two: How much RAM do I need to plan for Integration Services (SSIS)? Andy Leonard (<a href="http://sqlblog.com/blogs/andy_leonard" target="_blank">blog</a>|<a href="http://www.twitter.com/AndyLeonard" target="_blank">twitter</a>) did manage a succinct response: All of it! I, on the other hand, am not known for being succinct, so deferred the question for this post.</p> <p>Andy is right that SSIS wants as much memory as you can give it, which can be problematic if you’re executing an SSIS package on the same box as SQL Server. On the other hand, there are benefits to executing the package on the same box as well, so there is no one-size-fits-all solution. And the solution for one data integration scenario might not be the right solution for another data integration scenario. A lot depends on what CPU and RAM resources a given server has and how much data is involved. In order to know how much horsepower you need, you’re going to have to do some benchmark testing with packages. Here are some good resources for SSIS if you’re concerned about memory:</p> <ul> <li><a href="http://sqlcat.com/sqlcat/b/top10lists/archive/2008/10/01/top-10-sql-server-integration-services-best-practices.aspx" target="_blank">Top 10 SQL Server Integration Services Best Practices</a> from the SQL Customer Advisory Team (<a href="http://sqlcat.com" target="_blank">blog</a> | <a href="http://www.twitter.com/sqlcat" target="_blank">twitter</a>): This article provides an overview of best practices (as the name implies!) and includes links to information about using performance counters to monitor resource usage and about optimizing the Lookup transformation, which is one of the big memory consumers in SSIS. </li> <li><a href="http://msdn.microsoft.com/en-us/library/cc966530.aspx" target="_blank">SQL Server 2005 Integration Services: A Strategy for Performance</a>, a whitepaper by my friend, former colleague, and co-author of <a href="http://www.amazon.com/Business-Intelligence-ebook/dp/B004OR1XYC/ref=sr_1_2?ie=UTF8&amp;qid=1315496545&amp;sr=8-2" target="_blank">my first book</a>, Elizabeth Vitt. Although it was written for SSIS 2005, the principles related to tuning packages and how to benchmark still apply. The significant changes between SSIS 2005 and SSIS 2008 with regard to performance were improvements in thread management and in the <a href="http://www.sql-server-performance.com/2009/SSIS-New-Features-in-SQL-Server-2008-Part2/" target="_blank">Lookup transformation</a>. </li> </ul> <p>Is there a rule of thumb for deciding how much memory you’ll need for SSIS? Well, no less than 4 GB per CPU core is a good place to start. But if that’s not possible, you certainly want to have memory that’s at least two or three times the size of data that you expect to be processing at a given time. So if you’re processing 1 GB of data, you’ll want at least 2-3 GB of memory and, of course, more memory is even better!</p>PASS Workshop “Creating a BI solution from A to Z” - Interviewhttp://sqlblog.com/blogs/davide_mauri/archive/2010/09/16/pass-workshop-creating-a-bi-solution-from-a-to-z-interview.aspxThu, 16 Sep 2010 22:16:27 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:28820manowar<p>If you’re still unsure if the workshop I’ll deliver at PASS Summit the next November - “Creating a BI solution from A to Z” - is a good choice for you or not, you can get some more details reading the brief interview here:</p> <p><a title="Permanent Link to 2010 PASS Summit Post-Con Preview - Davide Mauri" href="http://www.sqlpass.org/Community/PASSBlog/entryid/187/2010-PASS-Summit-Post-Con-Preview-Davide-Mauri.aspx">PASS Summit Post-Con Preview - Davide Mauri</a></p> <p>Where you’ll find answers to the following questions:</p> <ul> <li>Is there an audience that would benefit especially from this session?</li> <li>After having attended your seminar, what are two or three things that an attendee will be able to take back to the office and put to use right away?</li> <li>What background should attendees ideally have to be fully prepared for your seminar?</li> <li>What experience are you, as a speaker, bringing to this session?</li> </ul> <p>That may help you to take your final decision. More than 20 people already decided to come: you’ll be in good company! :)</p>PASS Summit 2010 – BI Workshophttp://sqlblog.com/blogs/davide_mauri/archive/2010/06/30/pass-summit-2010-bi-workshop.aspxWed, 30 Jun 2010 12:01:10 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:26596manowar<p>PASS Summit 2010 Pre &amp; Post conference <a href="http://sqlpass.eventpoint.com/PrePostConferenceSessions">are out</a>! This year I’ll deliver the “<a href="http://sqlpass.eventpoint.com/topic/details/TF1588">Creating BI Solution from A to Z</a>” seminar in which, as the title implies, attendees will see how to create a BI solution starting from scratch. Going through the dimensional modeling and the creation of the Datawarehouse, the implementation of the ETL process with SSIS, the creation of cube with Analysis Services and reports with Reporting Services with, if time permits, also a glance at PowerPivot, attendees will get a solid ground on the whole process that drives the creation of a BI solution.</p> <p>The workshop will also show all the best practices and the best methodological approach matured in more than 5 years of working in the BI field.</p> <p>If you’ll be working on BI or you already have a created an initial BI solution and you want to be sure you’re following the right path, this is workshop is for you!</p> <p>Don’t miss it!</p>SSIS Logging - The PASS Sessionhttp://sqlblog.com/blogs/davide_mauri/archive/2009/11/04/ssis-the-pass-session.aspxThu, 05 Nov 2009 00:51:27 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:18550manowar<p>Just finished my session here at PASS. Thanks a lot to those all who came, and thanks for the great feedback.</p> <p>I’ve made a promise that I’m going to release also the reports I showed in the demos asap, and I’ll do my best to do it, since so many of you are so interested in it.</p> <p>Meanwhile, if you wish, you can rate my session here:</p> <p><a title="http://speakerrate.com/talks/1704-instrumenting-monitoring-and-auditing-of-ssis-etl-solutions" href="http://speakerrate.com/talks/1704-instrumenting-monitoring-and-auditing-of-ssis-etl-solutions">http://speakerrate.com/talks/1704-instrumenting-monitoring-and-auditing-of-ssis-etl-solutions</a></p> <p>And of course, if you have questions, just get me there and stop me: I’ll be more than glad to give you more information!</p>A week in Seattlehttp://sqlblog.com/blogs/davide_mauri/archive/2009/10/30/a-week-in-seattle.aspxFri, 30 Oct 2009 20:02:00 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:18392manowar<P>Tomorrow morning I’ll fly to Seattle for the usual PASS Community Summit. I’m really happy for the next week. Even if this period is really full of work and a week away means that at my return I’ll have *really* busy times, the PASS Conference is always a nice break: meeting old and new friends, learning new things, enjoying beers and cocktails and dinners, and, of course, plan and look ahead into the future. Yes I’m really looking forward to be there.</P>
<P>In addition, this year, if you’re into Business Intelligence or Data Integration you’d better not miss my session, dedicated to some of the most common problems and requirements: logging, monitoring and auditing.</P>
<BLOCKQUOTE>
<P><STRONG>Instrumenting, Monitoring and Auditing of SSIS ETL Solutions</STRONG></P>
<P>Have you ever been struck by the lack of detailed logging available within SQL Server Integration Services (SSIS) packages? If the answer to this question is yes, then you are probably a seasoned Business Intelligence solutions builder. The packages that make up your solutions probably contain extra code to facilitate effective monitoring, control and profiling. However, increasingly, SSIS is being used to unify data found on disparate yet related transactional systems. The batch processes that perform this activity thus become a vital part of the information processing ecosystem. These need to be monitored and profiled in the same way as formal BI solutions but often the resources available are much more meager. What do you do when something goes wrong and the functionality of the packages is not transparent via internal state reporting. This session will introduce you to an improved version of DTExec. By passing the appropriate state flags it can reveal the internal activity of packages without having to make any changes to the packages themselves.</P></BLOCKQUOTE>
<P>Here’s a sneak peak of what you’ll be able to obtain from your SSIS Packages, after coming to my session, without the needing to write a single line of code or change existing packages:</P>
<P><A href="http://sqlblog.com/blogs/davide_mauri/image_58E3B274.png"><IMG style="BORDER-BOTTOM:0px;BORDER-LEFT:0px;DISPLAY:inline;BORDER-TOP:0px;BORDER-RIGHT:0px;" title=image border=0 alt=image src="http://sqlblog.com/blogs/davide_mauri/image_thumb_468AC5E5.png" width=244 height=168></A>&nbsp;<A href="http://sqlblog.com/blogs/davide_mauri/image_63B06DEF.png"><IMG style="BORDER-BOTTOM:0px;BORDER-LEFT:0px;DISPLAY:inline;BORDER-TOP:0px;BORDER-RIGHT:0px;" title=image border=0 alt=image src="http://sqlblog.com/blogs/davide_mauri/image_thumb_5561CF32.png" width=244 height=164></A> <A href="http://sqlblog.com/blogs/davide_mauri/image_6AFC07CF.png"><IMG style="BORDER-BOTTOM:0px;BORDER-LEFT:0px;DISPLAY:inline;BORDER-TOP:0px;BORDER-RIGHT:0px;" title=image border=0 alt=image src="http://sqlblog.com/blogs/davide_mauri/image_thumb_345E36BE.png" width=244 height=164></A> </P>
<P>See you there!</P>Presenting at the PASS Application Development Virtual Chapter 8 Sep!http://sqlblog.com/blogs/andy_leonard/archive/2009/09/03/presenting-at-the-pass-application-development-virtual-chapter-8-sep.aspxThu, 03 Sep 2009 11:00:00 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:16544andyleonard<P><STRONG>Business Intelligence at REST<BR>September 8th at 1:00 PM Eastern<BR>Presenter: Andy Leonard</STRONG>&nbsp;</P>
<P>I am honored to present Business Intelligence at REST to the <A href="http://appdev.sqlpass.org/" target=_blank>PASS Application Development Virtual Chapter</A> on 8 Sep 2009!</P>
<P>This is going to be a fun presentation for me. I'm going to talk about my weather station and some business intelligence I've wrapped around the weather data collected from it. It doesn't stop there, I've turned <A href="http://twitter.com/AndyWeather" target=_blank>AndyWeather</A> into a bit of a social bot! I'll talk about my application that interfaces with the Twitter API - a REpresentational State Transfer (REST) interface.</P>
<P>I walk through the idea, the hardware, and software. Visit the <A href="http://appdev.sqlpass.org/" target=_blank>website</A> Tuesday, join the meeting, and I hope to see you there!</P>
<P>:{&gt; Andy</P>Speaking at Midlands PASS Chapter in Columbia SC!http://sqlblog.com/blogs/andy_leonard/archive/2008/09/26/speaking-at-midlands-pass-chapter-in-columbia-sc.aspxFri, 26 Sep 2008 20:48:00 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:9112andyleonard<P>I finally get to meet <A class="" href="http://blogs.sqlservercentral.com/brian_kelley/default.aspx" target=_blank>Brian Kelley</A> in person!</P>
<P>I'll be presenting Incremental Loads, Change Data Capture, and SSIS 2008 to the <A class="" href="http://www.truthsolutions.com/midlandspass/" target=_blank>Midlands PASS Chapter</A> in Columbia, SC on Thursday 2 Oct 2008! If you're in the area and read this blog, stop by and say "Hi".</P>
<P>:{&gt; Andy</P>