Curious..about datahttps://curiousaboutdata.com
One data professional's adventures with data..and beyondThu, 01 Feb 2018 13:27:58 +0000enhourly1http://wordpress.com/https://s2.wp.com/i/buttonw-com.pngCurious..about datahttps://curiousaboutdata.com
DBA best practices..from the DBA from Heavenhttps://curiousaboutdata.com/2018/02/01/dba-best-practices-from-the-dba-from-heaven/
https://curiousaboutdata.com/2018/02/01/dba-best-practices-from-the-dba-from-heaven/#respondThu, 01 Feb 2018 12:19:21 +0000http://curiousaboutdata.com/?p=3513Continue reading DBA best practices..from the DBA from Heaven]]>I met Tom Roush for the first time around 9 years ago – at a PASS Summit. If I recall right – Kendra Little introduced him to me as a colleague from Microsoft and a friend of hers. I met him at several summits following that – found him a kind person with extraordinary cheer and warmth. Our friendship really took off when I started to read his stories. I am an aspiring writer myself, and hope to be one full-time some day. I loved his style of narration and the emotional appeal in them. One of my favorite ones was the one with three christmas trees, which was shared by someone on facebook. I posted a comment there on how much I liked the story. He responded and sent me a friend request. I write little story-like stuff on facebook from time to time – mostly real life happenings that have touched me in some way. He would send me private messages on the story. ‘I liked how you explained that’, ‘this line shows emotional maturity’, ‘on this line you have said this, but perhaps you meant that’…and so on. He did this entirely on his own – somehow sensing that the budding writer in me was so thirsty for feedback. I learned from some mutual friends of his illness. I never asked him about it as I thought it was personal to him. He shared it on his own one day, and sent me link to the blog where his family wrote on his health and progress. I read it regularly. My mother had passed because of cancer. I knew from reading that his illness was serious and he had limited time.
Sometime during the fall last year – I got an offer to author a book from Apress publishers. It wasn’t exactly a book to write – but more like a series of interviews with data people who were very good at what they did. The choice of people to include in the book was up to me. Tom came to mind immediately. But I did not know if his delicate health would be able to stand the rigors of the hour long interview that the book demanded. So I asked him if he’d be willing to write the answers for me instead. I needed material to fill 10 pages – so there would be considerable typing involved. Tom sent me the answers. It was short of what I needed – so we decided to do another round – after his chemo last week. He said how excited he was to get his name on a ‘technical book’ and promised that he would do it. But that was not meant to happen. Below are Tom’s answers – for your reading. I sorely regret not being able to put them into the book and have it published while he was around. But we got it started, and now, people will read it for sure. And if you are a DBA, double check if you are doing what he said. Because doing nothing is unacceptable. Even if you are battling cancer. The rest of us have no excuse. NONE EVER. Bill Gates said of Steve Jobs, on the latter’s passing – that knowing him was an ‘insane honor’. I want to say the same – knowing Tom was an ‘insane honor’. I am honored to publish as below, Tom’s last interview.

1 Describe your journey into the data profession.

[Tom Roush: ] photography – first photojournalism, then freelance, created database to keep track of business – eventually used those skills to transition into IT

2 Describe a few things you wish you knew when you started your career, that you know now and would recommend newcomers to this line of work know?

[Tom Roush: ] I came into it from a photography background – where there’s the right way and every other way to do stuff – understanding in IT that there were SO MANY right ways to do something was really, really hard.

My path was something like this:

Health insurance company

application support

application developer

application administrator for group of 5 people

move to Microsoft

application support/administrator/developer (but for group of about 1000 people globally – comprising 10 databases (there’s more to this) all supporting MSN

report/graphing developer

Move to Getty Images

SQL developer

SQL dba dev/test/staging/load

SQL dba dev/test/staging/load/production

Move to Avanade

SQL DBA dev/test/staging/prod

3 What is a typical day in your life as a professional?

[Tom Roush: ] this has been very interesting because there’s so much change in it. I used to be the sole dba, keeping about 140 servers running. This was too much, the work/life balance was completely off, and we ended up expanding to a follow the sun model and hired 5 other dba’s to help me, and we ended up with three in India, 1 in Manila, 1 in Buenos Aires, and me in Seattle. The Buenos Aires one transitioned to a fellow in Toronto. My role in this is constantly migrating from being a production Tier2 dba to being a production Tier3 lead dba – meaning I will occasionally write code, I get called on for some deep troubleshooting, but a lot of my job involves checking email and trying to keep track of who’s doing what and deconflict various tasks that are being done on the same server. So – a lot of my day is spent dealing with email from dev teams in India, my ops team in India, Philippines, and Toronto, and then solving the problems they’re dealing with or unblocking them. I also spend time passing on knowledge or training them. A tremendous amount of what I do has to do with training my team, learning about and overcoming cultural issues. This is tremendously important because the words we use are not necessarily heard the same way by the various people on the team. Example: for a time I was dealing with the culture of all the folks mentioned above, and they all had varying skill levels, different work ethics, different things that motivated them. (What motivates the team in Bangalore is radically different than what motivates the fellow in Manila.)

5 Describe a few things which any data professional should know as best practices?

[Tom Roush: ] do not try to reinvent every wheel you need.

Every problem you face will likely be a problem someone else has faced before.

Know that there are people who will want to help you if you are brave enough to ask.

Monitor, monitor, monitor – and know what to do when you discover something wrong.

When troubleshooting – start with the simple – but be prepared to go deep.

6 Describe a few things which any data professional should avoid as worst practices?

[Tom Roush: ] doing nothing (unacceptable)

No backups (unacceptable)

The 9 letters that can get you fired RPO/RTO/CYA

being a lone ranger. Definitely get involved with others – don’t have lunch by yourself if you can avoid it; take the time to get away

8 Describe your experience with cloud adoption.

[Tom Roush: ] we have moved many of our systems, in whole or in part, to the cloud, from iaas, paas, and so on.

My personal experience has been that my team does the work

9 What are some of your favorite tools and techniques?

[Tom Roush: ] tools? Frankly, ssms is what works and what I can use. Having the budget for tools I’d like to purchase has been an issue – so I end up either writing my own tools or finding tools/scripts out there. Those would be:

Sp_whoisactive

The SP_blitz family of scripts

Ola Hallengren’s scripts

My own scripts – I write code so it’s dynamic – meaning it knows which datacenter it’s in, which environment it’s in – code that’s written this way may, for example, not have backups running in the test environment, but does have them running in production. In essence, got a GPS on it. The code is identical on each server, the variables are dynamically generated values for each individual server.

[Tom Roush: ] flying in a sailplane, writing stories, walking by the beach, prayer, meditation, totally disconnecting from electronics (this is a struggle for me).

13 Describe your style of interviewing a data professional – what do you look for and what are some examples of questions you ask?

[Tom Roush: ] Conversationally – I use Brent’s interview questions with my own additions. They’re always open ended questions that are very specifically real life types of scenarios. I’ve been in the business long enough to be able to have a few. I look for flexibility in thinking, the ability to start with the basic questions and work toward the complex, I also insist that they explain things to me in a non-technical way – like they would explain to an elderly relative.

Numbered questions are below

If I give you a new sql server and tell you to set up backups, what do you do?

tell me something about sql that mystifies you

explain diff between a clustered index and nonclustered index.

multiple users report sql is running slow for the first time today – what do you do?

explain diff between simple/full/bulk logged recovery models.

when you have to work on a server you’ve never touched, what’s the first thing you do?

a vendor app has slow queries, what are some ways you can do to make them faster?

latest thing you learned the hard way about sql server.

how to you keep from running a query on the wrong server?

explain RPO and RTO (six letters that get you fired)

name two ways you can tell sql has restarted unexpectedly the night before.

situation: blocking, you get alerts, but you don’t know which db. What would you do to track this?

14 What are your contributions to community and why do you recommend people be involved iwth community?

[Tom Roush: ] I spoke at several SQL Saturdays, spoke at the SSWUG virtual conference for three years, blog, and I do my best to stay involved in community.

[Tom Roush: ] treat everyone with respect. Know that not everyone has gotten to where they are using the same method you used, everyone has different experiences – and strengths come from those.

]]>https://curiousaboutdata.com/2018/02/01/dba-best-practices-from-the-dba-from-heaven/feed/0diligentdbaSQL Saturday Nashville 2018https://curiousaboutdata.com/2018/01/15/sql-saturday-nashville-2018/
https://curiousaboutdata.com/2018/01/15/sql-saturday-nashville-2018/#respondMon, 15 Jan 2018 12:13:33 +0000http://curiousaboutdata.com/?p=3511Continue reading SQL Saturday Nashville 2018]]>I decided to go to the SQL Saturday at Nashville this year to get some good learning and networking in during frigid January. One of the reasons I wanted to go was that the line up for precons was amazing and I wanted to learn more on Azure particularly. I also wanted to submit to speak but that did not happen, for many reasons. I took the wednesday evening flight out of Louisville and got into Nashville late that night.
On thursday morning, after a light breakfast – I took a lyft ride to the school for a half day precon with Josh Luedemann called ‘Demystifying Data Movement in Azure’. We got to learn about PowerApps and Flow, Data Factory, and streaming data options in Azure. It was an interesting and informative 3 hour class. I went back to the lodging in the afternoon and spent my time reading and preparing for a presentation. There were already warnings out for a snowstorm on friday and I was beginning to get apprehensive regarding weather.

Friday morning arrived with news that school was closed for the day and there would be no precons. I was saddened as this was the main reason I had made the trip from Louisville. But it could not be helped – so spent most of the day at the lodging, reading and learning various stuff. Thankfully there were restaurants on ubereats that were still functioning – was able to get good food ordered in.

By Saturday the weather had cleared up – made my way into the school bright and early. It was cold but there was not much snow or ice around to worry about. There was coffee available for attendees – after getting some in made my way into the first class of the day – Azure Machine Learning 101 with Kathi Kellenberger. Kathi covered basics of AML and predictive analytics in a very down-to-earth manner. She mentioned things like being asked about ‘what was regression’ in an interview. I suspect all of us who claim to be playing around with ML must be prepared to face questions of this nature in future interviews. It was a timely warning to me personally to have those definitions handy as answers.

The class I had planned to attend next was cancelled – so I wandered into my friend and SQL Saturday co-organizer John Morehouse’s session on ‘How to put out Database fires’. John is a former firefighter, and he used many analogies in comparison to drive in the importance of best practices and doing database administration safe way. I thoroughly enjoyed the class.

The next class I went to was Chris Hyde’s session on ‘Python in SQL Server 2017’. I had already attended the same session at PASS Summit, but still felt that I could use a refresher – so went in again. Really enjoyed the simple examples and tips and tricks on how to get started with Python.

After a quick bite of lunch I walked into a panel discussion on cloud – with many talented folks including Bradley Ball, Josh Luedemann, Jason Horner, Hope Foley and Gareth Swanepoel There were many questions raised and answered with regards to cost, feasibility, cool new features and so on. It was very worth attending.

The next session was also a repeat one for me – ‘Data Cleansing with SQL and R’ – with Kevin Feasel. I greatly enjoyed it.

The last class of the day was ‘Twitter Data & Sentiment Analysis: Staring Logic Apps, Azure ML, Twitter Data, Power BI, and Python’ by Bradley Ball. Bradley has always ranked very high on my list of favorite speakers – he is a total natural on stage. In this class he explained using various tools to scrape twitter for movie related comments, cleaning that data and ultimately presenting it in PowerBI as a report on how the movie performed with public. It was very well researched and incredibly fascinating class for me.

After the class I touched base with several friends whom I had not seen in a while – said my good byes and left early. Although the day long class for which I had traveled in for did not happen – I was able to get a very good day of learning and networking in. The event was well organized and there were a lot of happy attendees as well. Thank you to organizers, speakers, sponsors and attendees for a good show, and hope to revisit Nashville next year too!

T-SQL Tuesday is a monthly blog party started by respected SQL Guru Adam Machanic. I was honored with hosting this month’s party. I picked a topic that is important for me at this time of the year – setting learning goals. I was surprised and somewhat overwhelmed to see the number of responses. I learned a lot reading what every person had to say. Below is the summary.

1 Rob Farley talks of tuning learning to what customers look for – which is a good idea for most of us. He also explains being interested in certifications and Microsoft EdX course for data science. This seems to be a certification many people are interested in. (Rob may not need as much ‘learning’ to do it as some of us as he is one of those who passed MCM without a lot of prep

2 Andy Leonard talks of learning more on cloud based offerings. He also explains challenges with managing businesses, home-schooling children and various challenges it presents with managing time. Andy is considering getting up at 5 AM to find time to learn – I must confess that I went ‘wow’ reading that because am not a morning person and that speaks for so much dedication. There is no shortcut to hard work and here is a prime example.

3 John Deardurff talks of finishing what he started – John is the certification guru of #sqlfamily, he has every cert imaginable under his belt and has listed a few more to go, in addition to finishing a college degree. Again, nothing short of inspiring.

4 Bert Wagner talks of learning more DBA skills as a developer to architect solutions better. Bert has also included a fun little video talking of his goals, very worth watching and very creative. He mentions the importance of creating demo-s as a way of learning.

5 Blogging rockstar and my own personal inspiration to blog – Brent Ozar talks of learning and creating gamification techniques around technology design such as cloud deployment strategies. Gamification techniques are intended to leverage people’s natural desires for socializing, learning, mastery, competition, achievement, status, self-expression, altruism, or closure, or simply their response to the framing of a situation as game or play. (Yes I had to google that and read up on what it was!). I can’t wait to read more of what he comes up with in this regard and I’ve always found his ideas to be incredibly creative.

6 Michelle Haarhues talks of learning more on Microsoft Azure and passing related certification exams. She compares tech learning goals to fitness goals and likes learning at Starbucks compared to home. She ends her post with a well-worded crisp mission statement on her goal.

7 Jeff Mlakar talks of one of my favorite topics – time management. His post includes a very good video-lecture on this topic. It is very worth watching. Jeff’s goals include MCSE, blogging,speaking at various sql saturdays and networking.

8 Steve Jones, one of my best friends and mentors in the community talks of various challenges that come up with setting structured goals in general, following a book or a course to the end (his experience with powershell is almost exactly the same as mine), and tracking goals. Steve’s post resonated with me in many ways.

9 Deborah Melkin aka Deb the DBA talks of learning better automation of testing techniques and her own style of learning – visual and kinesthetic. She explains the importance of matching how we learn to our learning style and has also included a link on learning styles.

10 Chris Voss talks of completing his MBA program and focusing on skills he needs as a data architect. He mentions the importance of DBA fundamentals as well as using the SQL community slack channel.

11 Long time sql community member and security guru Brian Kelley talks of maintaining MCSE and CISA certifications, as well as doing the data science certification. He talks of the difference between just getting a certification and gaining knowledge as part of the process.

12 Garland McNeill talks of list of skills to gain as a DBA and the importance of working complex problems as a way of assimilating the learning.

13 Tracy Boggiano talks of being inspired by Andy Warren’s talk on ‘Building a Professional Development Plan’ at the summit. I’ve heard Andy deliver this talk too and have been similarly inspired. Her goals including getting better at unix, talking at more events (she has spoken at 18 sql saturdays last year – well done Tracy!) , and learning more on SQL Server 2017.

14 Marek Masko talks of earning MCSE on cloud platform, speaking at double the number of conferences than he did this year and attending local events in Poland. He also has a great list of books on database engineering that are worth considering for any of us.

15 Taiob Ali’s goals include Microsoft Big Data Certification,Architecting Azure solutions and learning more on powershell. He also wants to contribute more to #sqlhelp answers and stack exchange.

17 Melissa Connors from SentryOnewrites on learning more on documentation and communication – she includes links to courses on EDx related to this, it is a very interesting list. She also includes some pictures of getting to her goal of baking macaronis (yum!) and training her dog Trekker.

18 Samir Behara’s goals include technical certifications and learning more on cloud technologies. He has made an interesting analysis of correlating website traffic and speaking engagements, showing that speaking increases visibility to the blog as well.

19 Glenda Gable’s goals include getting technical certifications, learning more using Brent Ozar’s training, blogging and using MSDN for labs. Glenda is also interested in being partners for accountability towards our goals.

20 Todd Kleinhans wants to learn more on Linux, Hekaton and do the Microsoft Data Science program, in addition to a number of personal goals. Todd has a lovely habit of writing hand written notes of thanks or appreciation for any number of things and posting it on social media. He did one of this blog party too which I will share towards the end.

21 Former PASS Community Evangelist and BI Architect Cathrine Wilhelmson talks of her goals that include learning more BIML, speaking, blogging, teaching and researching. She wants to revisit her post in December too to find out how she did.

22 Lucas Kartwidjaja talks of learning more of AWS/Azure via online courses and also do more of cloud architecture.

23 David Hiltenbrand has many interesting things to learn on his list including CosmosDB, graph databases, Adaptive Query processing via several online courses. He also plans to attend SQL Saturday Nashville (hope to see you there David!).

24 Martin Catherall from down under talks of general trends around learning and the importance of learning something deeply.

25 Stuart Moore blogs about doing more networking, improving his writing skills, doing more automation (everybody should!) and getting better at marketing himself. I could see eye-to-eye with every one of those goals myself.

26 Doug Purnell has a very interesting link on tendencies when it comes to goal setting – there is a test there that seems informative (mine came up as ‘questioner’, which I am) – I guess one needs to read the book he recommends to find out how this helps in setting a plan and making it work.

27 Andrew Tobin writes on getting into a new job that is 100% sql server, learning more related sql server stuff online, blogging without ‘getting personal’, answering questions on Stack Exchange and doing videos on performance tuning. Good luck Andrew!

28 Shane O Neill uses a joke to explain the many things he has on his list to cover next year including Azure, CosmosDB, containers, python and so on. It is a really creative way of presenting a plan and fun to read.

30 Jason Squires talks of many things he wants to learn including XEvents, Powershell and Availability Groups – he also includes his goals to improve on blogging and speaking at online events.

32 My good friend and mega blogger Kenneth Fisher writes on his own ‘eclectic’ learning habits and includes cloud,t-sql,performance tuning and learning more on metadata. Ken recommends starting to blog slowly to build up momentum – a great tip for new bloggers.

Thank you to all of you for taking time to contribute. I like the suggestion made by Glenda Gable on being partners to help with accountability on our goals. If anyone feels up to this just leave a comment below, we can set up a slack channel or something similar to help each other.

This month’s T-SQL Tuesday is hosted by…er..yours truly. I picked a topic that usually comes up for me every December – learning goals for the next year. This is the first time that I am blogging about it.

Usually this review is accompanied by a strange combination of imposter syndrome, guilt and thoughts like ‘how about not having goals’,’who cares’,’I want to retire’,’I am never going to be as good as so-and-so’ and so on. In short, it has rarely been a pleasant experience. A lot of people me included, want to learn a lot in the little time we have. We underestimate other life challenges that get in the way, and also our own power to assimilate and actually use what we learn. We also have different degrees of focus and methods of learning that work for us and there is no one fit for all. This year I decided to be more pragmatic and take all of those things into consideration while making a learning plan. I took into consideration below:

1 I can devote about an hour a day to learning on a normal day – this excludes days when I am on call and have had a late night, or some kind of a personal stressful situation.
2 I learn best by in person classes, followed by podcasts and videos.
3 My areas of interest are wide – DBA work with BI/Data visualisation. I need to pick what I want to learn carefully.
4 I consolidate well when I apply at work(true for almost everyone), teach/present and blog.
5 I network better in small crowds and familiar places.
6 I usually walk for about an hour in the evening. I can use that time to listen to podcasts.
7 My ‘lab time’ is the time I spend actually putting what I learn to actual practice – is typically about 2-3 hours late Saturday afternoon.

Keeping all this in consideration, my plan is as below:What to learn:My areas of focus are as below:

1 Pluralsight will be my main tool to begin with since it is easy and affordable. The goal is to try and get in 1 hour of pluralsight training per day on good days.
2 I will be attending select sql saturday precons as a second option, particularly for Azure and Data Science related training. I’ve set aside a budget for this and will only be able to do those events that fall within the budget. Chances are that none may.
3 I will be watching PASS Summit recordings and Ignite recordings for additional info. I don’t consider either of these as in depth learning but more as informative sessions.
4 I will be using Edx for the Microsoft Professional Program in Big Data.
5 I will be attending TechOutbound (formerly SQL Cruise) in March and the PASS Summit later in the year for networking. Additional networking will be possible with sql saturdays if they happen – most definitely with my own sql saturday event in July.
6 I found a treasure of podcasts that are out there that I can listen to on my evening walk. My favorite site for them is sqldownunder, by Greg Low. But there are several others – list here. These are my insurance that I get *some* learning in even if all the rest of the stuff does not work out. Listening during exercising is easy and exercising is something I do with a passion.

How to apply:
1 I am somewhat limited in terms of how much I can travel, for many reasons. So, I will be presenting at several virtual chapters and also user groups that allow remote presentation. Right now the goal is 5 presentations, more depending on time.
2 I plan to blog at the rate of one blog post per week. I’ve come to this as a result of a lot of experimentation and this works well for me. Every other week will be a non technical post since I have a lot of non technical info to blog about.
3 I do not consider application of learning to work as something I can blog about, so not writing more on that.

I also plan to journal weekly on how I did in the past week and tune myself as I go.By ‘journal’ I don’t mean writing pages – just a simple excel sheet where I review what I learned versus what I wanted to learn. If I go for three weeks without meeting any goals I am doing something wrong and need to re adjust. Am hoping to keep my pace with this better and do better as I go. I think this is vitally important. It is not a bad thing that we failed to meet our goals but it may be bad if we never even considered tracking our progress through an entire year.

One of my favorite movie scenes is from ‘Gone with the wind’, where there is a garden clock with the lines ‘Do not squander time, it is the stuff life is made of’..right next to it is a cat sleeping blissfully. That sums up time management in general – you can’t force it. You have to take little steps, be consistent, and factor in all the other things that go into how you use your time. I hope to be better at it this year than I was in the years before.

This is my first opportunity hosting a T-SQL Tuesday and am super excited!!

T-SQL Tuesday is the brainchild of well respected SQL Guru and author of ‘sp_whois active’ – Adam Machanic (b|t). Adam rightly predicted that we all could benefit from ‘a recurring, revolving blog party’ with a new topic given each month – the party has been on since 2009 with great benefit to bloggers old and new.

This month’s T-SQL Tuesday will take place on Tuesday, December 12, 2017.

It is the end of the year…each person has their own way of calling it a year. For many it is time to take those leftover PTO days, enjoy the time with family and friends, and be grateful for the many gifts we are fortunate to have received. It is also a time when we have to ponder the year ahead and how we plan to use this time in the best manner possible. For many years I never consciously considered setting or pursuing learning goals. I just learned what I needed for my job. If there was a new version of SQL Server out – I went after learning new features in it. But today – it is no longer possible to be that simple. Also, it is very difficult to focus on anything unless you put it down as a goal. One of my favorite quotes by Zig Ziglar sums it up best ‘ A goal properly set is halfway reached’.
Learning more on just SQL Server is no longer enough. We need to learn other tools and technologies. There are many of them. There are 3 things to address to me when it comes to goal setting with technology and learning –

1 What do you want to learn? (specific skills and talents)

2 How and when do you want to learn? (methods of learning and timeline on learning)

3 How do you plan to improve on what you learned? (Putting it to use at work/blogging/speaking)

I will explain each of those in detail below.

1 What do you want to learn?

This varies depending on your line of work and where you want to go career wise. I will give a few examples.

If you are into learning about the cloud and hosting – you need to know what options are (on AWS and Azure, to begin with). Also on multiple other smaller/private hosting providers. You need to know how to transfer data/how much it costs to scale/can you turn it on and off as necessary…any number of things.

If you want to learn other non SQL database platforms you’d have to think about which ones are important to you – postgres, CosmosDB, DocumentDB or even MYSQL or Oracle.

If you plan to get into data mining and analytics – there are several things to learn in that area. I just started getting to intermediate level with R , and now we have Python that works just as well with SQL Server. You are also better off learning other skills that go with data mining – such as cleaning data, setting up the solution on an ongoing basis and so on.

In general it would be wise to narrow your focus down to your areas of interest and pick a few things – not too many but perhaps 2-3 things you’d like to focus on and get some depth of knowledge in.

2 How and when do you want to learn?

After you get those goals in, how do you plan to get the said training?

There are countless options, with time and costs to consider. The cheapest ones are Ignite videos (for free), Pluralsight subscription (30$ a month), EDx/Udemy courses (all reasonably priced).SQL Saturday precons (very reasonably priced day long training) as well as SQL Saturdays themselves(free day long training on saturdays). If you can afford it yourself or work at a company that pays for training – consider Tech Outbound (formerly SQLCruise) or PASS Summit.

There are networking goals to consider as well. I personally would never have thought of networking as a ‘goal’, am able to tweet or message most folks and talk to them, so what is the big deal? No. Meeting people in person is a whole different thing, and you never know what doors that can open. Networking goals can be like meeting 10 people new (some people set them that way), or catching up with 50 people you already know including 3 lunches with people who have most regard for. You’d have to consider where and how you are going to get those goals met. For some people, like me, this is not a numbers game – I’d just like to say am going to be at Event A, B and C and do my networking there. That is totally fine too.

3 How do you plan to improve on what you learned?

The primary application of knowledge is at work. You want to think of upcoming projects or opportunities to apply this knowledge. For most people this comes up at a performance review that happens early in the year. Many people are also not comfortable making it public. If it is not bloggable that is ok – but if it is bloggable do consider sharing it.

What are the chapter meetings, events you plan to speak at? If that is too much detail, consider how many of those you’d want to do.

This month’s T-SQL Tuesday is hosted by Ewald Cress, and the topic is ‘Folks who have made a difference’. This is an opportunity – and right around thanksgiving – to give a shout-out to people who have made a difference in our careers and in the data community. To me this list is so long that I would need several blog posts to write about every person on it. But am going to pick one person, and for a very good reason. That person is Andy Warren(b|t) – the founder of SQL Saturdays.
Back in 2008, I was actively running the local user group here at Louisville, and volunteering for PASS in several different capacities. I was always looking for other opportunities to bring training to the local community, that was more than one hour talks we had at user groups. Part of the reason I wanted to do this was because the vast majority of people I worked with and attended user groups were folks who had never attended any kind of formal training.Very few companies paid for them to go, and the idea of investing your own time to learn was very new, back then. You learned what your boss wanted you to learn, and if he gave you time/money for it. That was the general attitude, and one that struck me as very wrong, although I didn’t get very far trying to say that to most people. I met Andy at the networking dinner that he and Steve organized, at the summit. He asked me if I had heard of SQL Saturdays. They were free day long trainings, and could be held anywhere. It was interesting, but I was apprehensive. People attending training on saturdays? In a little town like this? I wondered about it and when I got home – I called my contact at a local school on a whim, just to find out if they had any classrooms they could lend to us for free on a saturday. The answer was a very emphatic ‘yes’, of course they could. Their rooms were small but they had 4 rooms, plus a nice big hallway, and small private rooms too. They’d like the exposure in community and actually wanted to do it. This looked like something that wanted to happen.
I registered on Andy’s site, picked a date and then looked to him for guidance on the rest. He guided me through it every step of the way, with constant reassurance that ‘if you build it ,they will come’. SQL SAturday #1 was event #23 for PASS. It was held at New Horizons, with 60 people in attendance(I expected about 40 people), 4 speakers(Arie Jones, Allen White, Kevin Kline and Dave Fackler) and two sponsors. I think we had about 3k as sponsorship money to run it. Almost every attendee said emphatically about how much they loved it. New Horizons wanted to host it yet again. The upcoming year, 2018, will be our year #10 running SQL Saturdays, in a row. Every year, we have been getting bigger and better. Hundreds of people have received free training, so many careers and lives have been touched.
I would never have done it if it was not for Andy talking me into it. I think every one of you who has been part of SQL Saturday Louisville need to be. Thank you Andy, for showing us what we could be.

]]>https://curiousaboutdata.com/2017/11/14/tsql-tuesday-96-folks-who-have-made-a-difference/feed/1diligentdbaSqlTuesdayDealing with disquiethttps://curiousaboutdata.com/2017/11/11/dealing-with-disquiet/
https://curiousaboutdata.com/2017/11/11/dealing-with-disquiet/#respondSat, 11 Nov 2017 23:33:51 +0000http://curiousaboutdata.com/?p=3357Continue reading Dealing with disquiet]]>Last week I listened to a podcast from one of my best friends in the sql community – Kendra Little. In this podcast Kendra talks of her encounter with anxiety attacks and how she dealt with them. When I listened to her honest, moving story – my mind was filled with thoughts on the many episodes with anxiety i’ve dealt with, each of its own kind. I was also moved by how many people in the community wanted to hear about stories like this to feel reassured on their own journey. So, here is one of my own. I am tagging a friend at the end of it ,hope he will tag someone else, and we can have a worthy collection of stories to refer to if we need help/reassurance.
Many years ago, I worked as a Senior DBA cum team-lead at a big firm. I was doing DBA work, and also helping my boss manage a team of six other people. My boss was a very kind,intelligent,generous man, one of the best I’d worked for.I greatly enjoyed my role and the work.A few years down the line, my boss got passed up for a promotion he richly deserved. After that his attitude and behavior changed.He started to be a no-show at important meetings,didn’t respond to emails, took time off without notice, and so on. One day, his boss decided to forward one of his meeting invites to me.I went, and filled in for him.The next day, I got more of his work. And then more. Soon, I was doing two people’s work, and working 12-14 hours a day. I wanted to speak to someone about this, but I kept putting it off with the hope that my boss would come around and get back to doing his stuff. I still really liked the job, and kept up with my needs for food, exercise etc too with the demanding schedule. At least I thought I did. One day, I started to feel some pain around my shoulders. I rubbed some balm on it and hoped it would go away. The next day, there was some tingling sensation in my feet, followed by some numbness and brief giddiness. I started to become very jittery and noise-sensitive. Somebody honking on the street would bother me for hours, with my hair standing on end and my heart beating extra loud. I had never had these symptoms before, and as Kendra mentioned – my life was going well according to me. So, what was wrong?

After a few days into this – as I was driving to work one morning, there was more loud honking on the busy street I had to use. My whole body was thoroughly shaken. Instead of going in to work, I drove myself to the ER – firmly convinced that I had some strange disease. They did all kinds of tests on me – brain mri, abdominal CT, heart exams, everything – pronounced me fine and sent me home in two days, with some medication to help me sleep better.

Two days later, I went in to the dentist for an unrelated problem with my wisdom tooth, still worried and firmly convinced that I had some unknown illness. While taking xrays of my teeth, the dentist said he noticed that my jaw bones were not aligned – a condition called TMJ. I asked him about my symptoms, and he nodded yes, I had TMJ , one of the major causes is stress and can be treated. This diagnosis was followed by a visit to a jaw specialist, some braces to wear and LOTS of relaxation therapy/counselling. After 2 months my ordeal was finally over. My TMJ still comes back now and then to remind me that I overwork or am not taking enough care of myself. But I know how to handle it now. Needless to say I moved on from that job shortly after.

Below are the lessons I learned from that episode and what I follow as practices for mental (and physical health), dealing with stress and anxiety.

1 Respect your body – Your body is an entity of its own. One of my friends likes to joke that you are its boss before 50 and it is your boss after. That is mostly true (you are never its ‘boss’, it just cooperates better when it is younger). Your body does not care how much you love your work or how long you want to do it. It is undoubtedly true that liking what we do leads to more mental happiness, it is not true that it is a safeguard against self care. Avoid the line ‘I love what I do’ as an excuse to skip meals/skip exercise/not getting enough sleep/not taking vacations or less family time. It is not worth it and can hurt you, a lot.

2 Connect with spirit – Kendra talks of this as going back to church/community where she could find succor/replenishment for spirit. I don’t particularly care for community worship, for many reasons. To me, connection with spirit happens with doing things that bring me joy – reading books I love, particularly books on women’s empowerment(‘Women who run with wolves‘ is my favorite), visiting book stores, antique malls, doing gardening, drawing or painting. Spirit is available in a place that is safe and free of rules, and those are the spaces for me.

3 Practice personal reassurance – I believe each person needs this in their own way. To me seeing art I enjoy , favorite pictures from vacations/with friends/family, or some phrases that resonate with me are very reassuring when I get anxious. I keep good art all around me, and phrases from books like ‘Tao of Pooh’ or ‘Jonathan Livingston Seagull’. I also invest money in enlarged prints of photographs taken on family vacations or sqlfamily reunions around me. They serve to enrich how I feel through the day. I change them around periodically but make sure that I see them – not just give passing glances but really ‘see’, standing in front of them, and re-live pleasant, happy moments.

4 Practice deep breathing and guided meditations. I recall a quote I read long ago – ‘Slow breathing is like an anchor in the midst of an emotional storm: the anchor won’t make the storm goes away, but it will hold you steady until it passes’. I practice it whenever and wherever i can, with my hand on my belly, where I feel my breath the best. It calms me down like nothing else. As a sound sensitive person, I love meditations that come with bilateral stimulation – a scientifically proven way to relax your brain. One of my favorites is here. Another awesome one called soft-belly meditation is here.

5 Understand your triggers and work with them – majority people who have anxiety have to deal with it periodically, it never really goes away fully. It teaches lessons in self acceptance that are invaluable. To me – I am triggered by loud noise, heavy traffic, noisy crowds and certain argumentative/demanding situations. And, as I learned from this particular anxiety episode, I need to find time for self reassurance. I don’t accept or work jobs that do not leave me time to balance these aspects of my life. As writer Stephen Covey says, that becomes like driving without finding time for gas. The car is going to stop, whether you like it or not.

Last but not the least, get help when your body reacts in ways you do not understand. I cannot stress this enough. Sometimes help is the first doctor you go to. Sometimes it takes longer and needs different types of doctors/healers/therapists/techniques. But persist. We live in an era where so much information is available online – search for information, ask friends on facebook or other social media on what they think. I am personally very grateful for the many people I have befriended because of the issues I have had – kind, sensitive, beautiful people who have taught me the value of life and importance of living in the moment. I hope to be the same to anyone who needs help with anxiety, stress or similar.

I tag one friend here – Tim Costello – to narrate his story. Tim, pass it on to someone else after you, and thank you.

“The most beautiful people we have known are those who have known defeat, known suffering, known struggle, known loss, and have found their way out of the depths. These persons have an appreciation, a sensitivity, and an understanding of life that fills them with compassion, gentleness, and a deep loving concern. Beautiful people do not just happen.”

– Elisabeth Kubler Ross

]]>https://curiousaboutdata.com/2017/11/11/dealing-with-disquiet/feed/0diligentdbaPASS Summit 2017 – the best everhttps://curiousaboutdata.com/2017/11/05/pass-summit-2017-the-best-ever/
https://curiousaboutdata.com/2017/11/05/pass-summit-2017-the-best-ever/#respondSun, 05 Nov 2017 22:28:50 +0000http://curiousaboutdata.com/?p=3317Continue reading PASS Summit 2017 – the best ever]]>I have been a PASS summit attendee for 14 years now. This year is my 15th. Every year is different – some years are better than others, some make you feel it wasn’t as good as usual. This year was an odd experience for me. Quite a number of my friends were not attending for various reasons, and there was a lot of content that I thought was not in my line of work. In short, I went because I always do but didn’t expect a whole lot. But, it didn’t quite turn out that way. Some of my highlights are as below:

Dressing up for Halloween: Had a good day at precon on Data Science essentials followed by a quiet dinner with a good friend and headed home to the airbnb which I shared with two other friends.I had bought a halloween costume along.Now, am not normally much of a costume person – I bought this costume (‘Maleficient’ the evil queen from Sleeping Beauty) at the very last minute – because I had some amazon points left and amazon in its own clever way thought I’d like this costume because I like Disney movies. I thought this summit was going to be very low key and decided that dressing up would probably make it more fun.So in went the costume in my suitcase, literally with prime wrapping intact.In the lodging – I saw my good friend Mickey dress up for halloween. She looked fantastic and the girl in me wanted to dress up as well. So I put on the costume, which actually fitted me right, and came with a lovely glowing staff. And off I went, first to the WIT dinner and then to the opening event. Everyone I met loved the costume and by the time evening ended I was beginning to tire of how many photos I had to pose for. The part of me that looks for attention had gotten more than her fair share during this one evening.

Making new friends: I am normally a somewhat reserved person. It takes me time to warm up and make new friends – part of the reason why I love the summit is because I know so many people there and they know me just by virtue of attending. There is no extra effort to stretch out and make friends. This year, I just decided to push the part of me that sinks into this comfort zone. I went out and made friends with people who I felt were worthy being friends with, and especially those who were looking to be part of the community. Those worthy contacts include Miyo Yuk, a data scientist from MIT, Swagatika Sarangi, immigrant from my home country and new speaker, and several others. I also met with new comers I was mentoring and had a very good conversation.

Asking for help/being mentored: Also a very difficult thing for me – when I do it I do it in very awkward ways that do not get desired result. This time I think I got it right. I got some awesome mentoring advice from two gracious ladies I have great regard for – Kathi Kellenberger on writing books, and Jen Stirrup on WIT related issues. I am glad to have reached out to both of them.

Learning data science: I have been blogging a while on some basics of data science. At the summit I attended some excellent sessions on how the data world is changing and evolving, what are the areas I can specifically focus on as a SQL Server professional looking to do more of data science related work, and who are the people I need to follow for getting there. I felt more energized than I ever thought I would that going in this direction would be the right thing for me, although it does involve a steep learning curve.

For all the reasons cited above and for many others, this summit was special. It has always been special but this year I felt a sense of true belonging with people in a very obvious way. The feeling was strongly like ‘this was it’. These are the people I am going to be with and grow old with through the rest of my career and my life. I am grateful and glad to know so many of them so very well. I am grateful that we are linked by a common worthy cause. If you are like me and reading this – looking for a community to belong and friends who support you/care for you – take heart, you have arrived. Just give it time and give it your whole hearted commitment. It will pay off.

True belonging is not something you negotiate externally, it’s what you carry in your heart. It’s finding the sacredness in being a part of something. – Brene Brown

]]>https://curiousaboutdata.com/2017/11/05/pass-summit-2017-the-best-ever/feed/0diligentdbaUnderstanding ANOVAhttps://curiousaboutdata.com/2017/10/09/understanding-anova/
https://curiousaboutdata.com/2017/10/09/understanding-anova/#respondMon, 09 Oct 2017 11:56:42 +0000http://curiousaboutdata.com/?p=3269Continue reading Understanding ANOVA]]>ANOVA – or analysis of variance, is a term given to a set of statistical models that are used to analyze differences among groups and if the differences are statistically significant to arrive at any conclusion. The models were developed by statistician and evolutionary biologist Ronald Fischer. To give a very simplistic definition – ANOVA is an extension of the two way T-Test to multiple cases.
Using the same dataset from previous post – the chicken feed one, let us analyze this further. The inference from the boxplot drawn in previous post was that the weight of the chickens is lowest when they are fed horsebean and highest when they are fed casein or sunflower. There were also lots of overlaps in weights. Our objective here is to determine if the average weight of chickens is significantly different based on feed, or is it more or less the same, which means the feed is really not that important to consider? We will define our null hypothesis(the statement to prove or disprove) as the average being the same and hence type of feed has no real consequence.Null Hypothesis:H0: There is no correlation between feed type and weights.Alternate Hypothesis:H1: There is significant correlation between feed type and weights.
What ANOVA does is calculate F statistic = Variation among sample means/Variation within groups. The higher the value of this statistic, the greater is the chance that variation among sample means is significant.
Running this simple test on chickwts dataset as below:

From this we can see that the F value is 15.365(Large is > 1), and the p value is really really small(to remember that ‘small’ is <0.05). So we can say with confidence that difference in weights between different feeds is way higher than difference in weight within same feed. In other words feed does appear to have an impact on weight. So we accept the alternate hypothesis.
Taking this one step further – what are the types of feed that cause significant weight differences? To understand this we perform what is called a Tukey’s HSD test, that compares each value to every other and helps us understand which pairs are significant.

It just takes a couple of lines of R code to do this – as below:

How to read/interpret the results of this test? Let us take the first line. The difference in weight with horsebean and casein is -163, which means casein is 163 points above horsebean. Since the p value is 0, the chances of this being significant are really small, as we can see with lower and upper limit values. So this is really not the pair we are looking for. Going down the list, the ones with significant p values (> 0.05) are meatmeal-casein, sunflower-casein, linseed-horsebean, meatmeal-linseed, soybean-linseed, soybean-meatmeal and sunflower-meatmeal. This can also be drawn in graphical form as below (i could not get R to shorten the text names but the ones beyond 0 are significant). The pairs with significant differences are the ones worthy of pursuit on which feed to adopt. Thanks for reading!

]]>https://curiousaboutdata.com/2017/10/09/understanding-anova/feed/0diligentdbaanovaanova1anova3Box-and-whisker plot and data patterns with R and T-SQLhttps://curiousaboutdata.com/2017/09/25/box-and-whisker-plot-and-data-patterns-with-r-and-t-sql/
https://curiousaboutdata.com/2017/09/25/box-and-whisker-plot-and-data-patterns-with-r-and-t-sql/#respondMon, 25 Sep 2017 11:53:37 +0000http://curiousaboutdata.com/?p=3191Continue reading Box-and-whisker plot and data patterns with R and T-SQL]]>R is particularly good with drawing graphs with data. Some graphs are familiar to most DBAs as it has been things we have seen and used over time – bar charts, pie diagram and so on. Some are not. Understanding exploratory graphics is vitally important to the R programmer/data science newbie. This week I wanted to share what I learned about the box-and-whisker plot, a commonly used graph in R – and one that greatly helps to understand and interpret spread of data. Before getting into specifics of how data is described with this plot, we need to understand a term called Interquartile Range. For each range or subset of data we are involved with (or rather the field we choose to ‘group by’), the middle value is called ‘Median’. The middle of the top 50% of values is called first quartile, and middle of the bottom 50% is called ‘second quartile’. Difference between first and second quartile is called ‘interquartile range’. A box-and-whisker plot helps us to see these values visually – and in addition to this also shows outliers in the data. To borrow a graphic from here – it is as below.

Now, let us look at the dataset on chicken weights in R with the help of this type of graphic.This is basically a dataset comprising of data on 71 randomly picked chickens, who were fed six different types of feed. Their weight was then observed and compiled. Let us say we sort this data from minimum to maximum weight for each feed.

You need the mosaic R package installed, which in turn has a few dependencies – dplyr, mosaicData,ggdendro and ggformula. You can install each of these packages with the command install.packages(“<package name”, repos = http://mran.revolutionanalytics.com&#8221;) and then issue below command for pulling up with B-W plot.

With this we are easily able to see that
1 Feed type casein produces chickens with maximum weight
2 Feed type horsebean produces chickens with minimum weight
3 Feedtype sunflower has some outliers which don’t seem to match general pattern of data
4 The distribution of weights within each feed type seem fairly symetric.
5 There are also many overlapping weights across feed types.

To get to some of the numbers accurately you can just say

favstats(weight~feed, data=chickwts)

If you wish to get some of these values in T-SQL (i imported this data into SQL server via excel) – you can use query below. Comparing the values visually with graph will show that they are similar.

SELECT DISTINCT ck.feed,MIN(weight) OVER (PARTITION BY ck.feed) AS Minimum, Max(weight) OVER (PARTITION BY ck.feed) AS Maximum,
AVG(weight) OVER (PARTITION BY ck.feed) AS Mean,
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY ck.weight)OVER (PARTITION BY ck.feed) AS MedianValue,
STDEV(weight) OVER (PARTITION BY ck.feed) AS SD
FROM chickfeed ck ORDER BY feed;

Results are as below:

The math for Q1 and Q3 are a little complicated in T-SQL, I was unsure if it was worth doing since that is not the point of my blog post. You can find info on it here

As we can see, the numbers tie up regardless of which way we do it. But it is much harder though to find patterns and outliers using code. The graph is undoubtedly more useful in this regard.

In the next post we will look into applying some analysis of variance (ANOVA) to examine if the difference in weights across feed types is really significant to arrive at any conclusion on nature of the feed. Thanks for reading!