voiceofthedba's podcastThu, 30 Jul 2015 01:00:00 +0000Sun, 02 Aug 2015 18:49:48 +0000Libsyn WebEngine 2.0http://www.voiceofthedba.com
enhttp://www.voiceofthedba.comhttp://static.libsyn.com/p/assets/c/e/c/7/cec74f5a804d1212/soapy.pngvoiceofthedba's podcastdatabases,sqlservernoThe Security PayoffThu, 30 Jul 2015 01:00:00 +0000I wrote a few months agoabout United offering rewards for peoplethat discovered security issues in the United Airlines software. Not the plane software, thankfully, but in their customer facing IT systems. Apparently a few people discovered flaws and were recently awarded frequent flyer miles, a couple of which received 1 million miles. That's a nice bonus for some people, though I hope the end result of this is that United builds better security into their software and learns to code better. Certainly other manufacturers have programs that have helped in the tech world. Microsoft, Google, and others have programs that pay bounties for flaws that are discovered and reported.

However I wonder if this isn't something that should become an accepted practice. I don't necessarily want regulation here, though I would prefer our regulation not prevent the research and exploration of security issues. Imagine, however, if we had an accepted, known process for finding flaws in all software. Maybe this would be like a standard process, like reporting spam to the webmaster@ address for websites. If someone finds an issue, they let the company know. The company has a bounty of some sort, perhaps a token reward, and a limited time to fix the issue. One the time passes, the individual is free to disclose the issue to the public, the problem can be publicly discussed and analyzed. More importantly, the company now has liability for any data loss or productivity issues.

]]>03:33cleansecurityHiring for CultureWed, 29 Jul 2015 01:00:00 +0000I've written in the past that I think that it's important the hire people that get along with your group first, and with technical skills second. It's not that technical skills aren't important, but I think that fitting in with a team is more important. We can teach someone to be a better developer. We can't easily teach them to be a person that others like to work alongside. Certainly, however, the person needs to have the technical skills needed for the job.

When I was interviewed in college, the hiring manager told me there were two important tests to pass: Would he drive cross country with me and would he go out for a drink with me after work? That doesn't imply that those are the most important (soft) skills for a candidate. It's that all the other, more quantifiable skills were similar in all the candidates. The ability to get along socially was a differentiating factor.

]]>03:08cleancareerVisualizing in SpaceTue, 28 Jul 2015 01:00:00 +0000One of the really interesting things that we are able to do as developers and DBAs is help find ways to analyze data for our clients and customers. Many of us might not have expertise in the actual areas that are represented by our data, the industry specific knowlege, but those of us that will be the most successful in the future at our jobs will be able to work with the people that have that specific knowledge and need help to implement the analysis.

That will take some practice. I'd encourage many of you to start playing with some sort of data that you are interested in and try to perform some analysis. Whether it's useful or just for fun, you will be gaining some skills in both analysis and presentation that will serve you at some point.

]]>01:55cleandata,visualizationWhat We Want and What We'll Pay ForMon, 27 Jul 2015 01:00:00 +0000I'm sure many of you see surveys that note IT job growth is up 30%. Or that executives want to hire 20% more people. Or that the market dictates that salaries will be going up 12%.

Those are exciting numbers, and I see them too. However I saw a great post that summed up the fact that those are just quotes, perhaps even desires, but not necessarily realities. This answer notes that what we want and what we'll pay for are often two different things.

I read a piece recently that noted Stephen Hawking, Elon Musk, and Bill Gates have warned us about artificial intelligence. They have been quoted as this is potentially something that the human race needs to be cognizant of. However, I wonder.

]]>01:43cleanintelligence,artificialThe Evolving DBAThu, 23 Jul 2015 01:00:00 +0000It's interesting to think about the role of the DBA and how it might evolve in the future. I regularly see predictions about the DevOps movement, and increasing ability of fewer staff to manage larger scales of systems (more systems or more data). There is also the fundamental way in which larger "cloud" type providers (inside your organization or third parties) can include features or services that reduce the need for DBAs.

However I ran across a statement that I think certainly provides some view into how the DBA might be affected in the future. The statement was: "The more an organization values its data, the more the DBA is under pressure to provide access to this data." That includes all the people working with and managing data, whether developers, sysadmins or dedicated DBAs. They must ensure the data is available and useful to clients.

]]>02:12cleandatabases,dbaThere Are Always ConstraintsTue, 21 Jul 2015 01:00:00 +0000Many of us have spent years learning about technology. I suspect that many of you might enjoy these pieces I write, but you visit SQLServerCentral for the same reason I started building it: to become better at your jobs. Many of you work regularly to improve your ability to make SQL Server perform as well as possible.

I'm also sure many of you have had to deal with what you see as "bad decisions" from your management at some point. I'm sure many of you have felt that if we did something a little differently, or spent a bit more money (or time), we could have built a better application. That may be true, though I've found over the years that technologists sometimes don't make better decisions than project managers. We often have different priorities, but also different views on what success means.

]]>02:03cleansoftware,development,careerPresenting DataMon, 20 Jul 2015 01:00:00 +0000Many of us that develop or manage database systems are concerned with the actual bits and bytes that compromise data. However our clients and customers are more interested in the information, in gaining knowledge from the numbers, strings, and dates that are kept in our database tables.

I really think that one of those things that can truly allow a developer or DBA to show their employer they are valuable to the organization. Employees prove this when they can retrieve information in a way that clients find valuable. Not that we, as the technical people value, but in the ways that clients find valuable.

]]>02:08cleanwriting,visualization,databases,query,tsqlContinuous Delivery In Real LifeThu, 16 Jul 2015 01:00:00 +0000I've been writing and presenting onContinuous Integration(CI) andContinuous Delivery(CD) for a few years now. In that time I've encountered a lot of excitement and a good deal of skepticism as well. Many people want to know more and more, who is actually using this stuff? It's nice to talk about Spotify and Netflix, but their problem domain, resources, and above all, youth, are something few organizations deal with.

What about Microsoft? They're a large organization, fairly mature (now 40 years old), and have their own levels of bureaucracy to manage. However they're fundamentally changed the way they perform development across the last decade and a half. Starting in the beginning of this century, they moved to make secure coding a priority. In the last few years, they've also started to truly speed up their development process with CI and CD, culminating (now) with an extremely quick Windows 10 development pipeline.

]]>03:54cleansoftware,development,integration,delivery,continuousJump InWed, 15 Jul 2015 01:00:00 +0000The SQL Server community is amazing. We help, share, bond, and care for each other in a way that is very rare for a large, disparate set of technical professionals. I see and hear validation of this constantly from people that participate in their first SQL Saturday and are amazed by the experience.

I think one of the amazing things about the #SQLFamily is that we do jump in and help each other. I rarely see the RTFM answers to questions, or ganging up by multiple people to pick on someone's ignorance. In fact, I see the opposite. Most often the SQL community calls out and refuses to tolerate unprofessional behavior.

There are any number of phrases that implore us to use bits and bytes, pieces of information that lead us to better decisions. However, can the data ensure we make the best decisions? Do the models we use get better and better over time? It's hard to tell.

]]>02:38cleandata,mining,analysisI Feel Like a MagicianMon, 06 Jul 2015 01:00:00 +0000I ran across this comparison of SQL Server to PostgreSQL. It's written from the point of view of a PostgreSQL developer, who certainly doesn't like the Microsoft product much, with no shortage of complaints. Whether you agree or not, I do think there are a few valid points.

However one of the quotes that really caught my attention was in the section on converting dates. The author says that "MSDN provides a table of these magic numbers.", referring to the arcane and completely unintuitive format codes that we use with CONVERT(). Fortunately FORMAT() was introduced in 2012, and simplifies things, but still has issues and limitations.

]]>02:40cleansql,serverSSMS MaturityThu, 02 Jul 2015 01:00:00 +0000I've used a lot of tools with SQL Server over the years. We had a variety of individual tools from Microsoft for SQL Server 4.2, including theisqlwquery editor that I used for years. I wrestled with the Enterprise Manager MMC plugin and eventually moved away to use the Embarcadero suite to work with SQL Server before coming back to Microsoft's SSMS in SQL Server 2005. I've had a love/hate relationship with that tool ever since, though theRedgate suite of extentionshas certainly made life easier.

However Management Studio (SSMS) hasn't really evolved in quite a few years. It seems that the enhancements and additions that have been made with each version have been minimal, sometimes barely working and rarely improved across versions. It's been disappointing that relatively few resources have been expended on SSMS, despite the regular evolution of SQL Server every 2-3 years. I suspect some of the problems were the pressure to release the core parts of SQL Server first, and link SSMS to the server product, despite the fact that it really needs to support multiple versions.

]]>03:19cleanmicrosoft,sql,server,ssmsTraining Computer ScientistsWed, 01 Jul 2015 01:00:00 +0000We work in an interesting industry. While I think you can certainly enter many other professions, electrician, plumber, artist, etc. without any formal schooling, programming allows you to earn a very good living with little physical effort. You can also work in any location, even from home. I think the opportunities and requirements open this field up to many more people than previous occupations in the past.

Of course, plenty of people still go to school to learn about computers, though I'm not sure I'd recommend that to people if you want to work in this business. I think you can teach yourself lots of the skills you need, and I think companies are really starting to realize that a college degree doesn't correlate with a strong technology worker.

]]>02:24cleancareerThe Importance of Our WorkTue, 30 Jun 2015 01:00:00 +0000My wife has often uttered a particular saying during stressful times in her career. She spent 20 years working in computer telephony and speech, often in sales, and would remind her colleagues that they "weren't saving babies" as they worked with customers.

It was a reminder that most of our work isn't, ultimately, that critical to the world. It matters for our businesses to success, for our careers to move forward, but most of the time we should keep some perspective on the value of the time we spend at work versus the rest of our lives.

]]>02:09cleancareerThe New DBAMon, 29 Jun 2015 01:00:00 +0000What does DBA mean? It's traditionally meant Database Administrator for technologists. However it's also been a synonym for a database developer, or even anyone that has raised privileges on a database system. The term has been expanded to people designing databases, and usually includes the BI developers or analysts that work with database related technologies.

In short, it's been a vague term that means different things to different people. However as I look forward, I think there's another interpretation I have. The people functioning as DBAs, are the Data Bridges to Applications. We take the data and find ways to ensure it's useful to applications. We make sure it's available. We connect systems together. We do the dirty work of cleaning data. We secure the bridge.

]]>02:16cleandatabases#SQLNewBloggerThu, 25 Jun 2015 01:00:00 +0000In April, Ed Leighton-Dick set up a challenge to motivate new bloggers to begin writing about their knowledge of SQL Server. This is a passion of mine, and I believe this is one of the ways you can truly help interest potential employers in your work. It's allows them to perform some due diligence, and I've had success in my career with employers being impressed with the knowledge I've shared through blogging.

Ed made note of a new Twitter hashtag, #SQLNewBlogger, and there were a few people that participated in the challenge. However, there weren't enough. I was really hoping to see more people join in, start blogging, and showcasing their knowledge. Therefore, I'm making an appeal here.

]]>04:57cleanblogging,careerSkill Supply and DemandTue, 23 Jun 2015 01:00:00 +0000Across my career, I've seen many managers and executives promote and press for IT groups to consider alternative technologies. The mass media has often promoted new platforms and languages as a more efficient way to build software. Some of their claims might be true, but I think there's a much bigger factor than technology in how well your software is written.

Your employees' skills are probably the most important influence on the quality of your systems. The technology doesn't matter if the staff doesn't understand it. I think it's even more important to be sure that you can replace your staff. People will come and go in your organization.

]]>02:52cleancareerFive Minute RefactoringMon, 15 Jun 2015 01:00:00 +0000I've listened to, and been a part of, no shortage of arguments about the technical merits of a particular programming solution. I've seen two (or more) developers argue about the way to solve a particular problem for hours, even days at times. The Internet is full of discussions and debates in forums on the "best" way to write code for all sorts of specific issues.

These arguments are sometimes referred to as religious wars for the devotion and belief that each proponent has in their solution. I tend to try and avoid becoming embroiled in any sort of zealotry as I am always cognizant of the fact that I need to be effective. I need to get things done and can't spend a lot of time arguing about a solution, whether for or against it. i need to get work completed.

]]>02:49cleansoftware,developmentThe Rename GameFri, 12 Jun 2015 01:00:00 +0000As I've been talking about database development, Continuous Integration, and Continuous Deployment, I keep hearing complaints and concerns about the problems that occur when we make database changes. In particular, I often hear about breaking changes made to databases, that require some manual scripting work.

It seems that the most common breaking change is adding a not null column to a table with existing data, but certainly there are other problematic issues. The other common change is renaming objects. In the past, I've rarely renamed anything once I've deployed tables to production, but I've run into quite a few developers that note they regularly change names. This week I'm wondering how common this is for most of the data professionals out there.

]]>01:59cleandatabasesPractical HekatonThu, 11 Jun 2015 01:00:00 +0000I got into a debate recently with someone about the In-Memory OLTP tables (formerly Hekaton) feature that was released with SQL Server 2014. This person asked if anyone was actually using the technology, given all the limitations that were placed on these structures in this first release.

I've had quite a few people actually get upset with Microsoft releasing a "feature" that doesn't just work as a replacement for the regular tables in SQL Server. They feel that Microsoft should have spent more time working to make the transition to these tables seemless, perhaps not releasing in-Memory tables until SQL Server 2016.

]]>02:31cleansql,server,inmemory,hekaton,oltpArchive Every DayTue, 09 Jun 2015 01:00:00 +0000I regularly see questions posted online from people struggling to manage a large archive process. Often the inserts into an archive table and the deletes from the source require large amounts of resources that strain systems and block other activity. These types of processes can also require lots of transaction log space, something many people forget about.

The best solution, in my mind, is to archive regularly. Decide how much data you need and then trim your working set back to that level on a regular, preferably daily, level. That way you limit the amount of resources you need at one time.

]]>02:38cleanarchivingWhen Should We Be Cleaning Data?Mon, 08 Jun 2015 01:00:00 +0000I was reading Grant Fritchey's update from the Business Analyst Conference and noticed this quote: " There were lots of sessions on the need to clean data, which implies that we need to do a better job at data collection in order to support this wild new world." Grant was talking about the fact that many of the sessions seemed to imply that it's mostly the processes around data that cause issues with data, not the people.

However is that really what we should do? Do we want to do more and more cleansing at the point where people enter data? I'm not so sure that's the case. The more that I deal with various applications and forms, the less I want to see too many fields and values required. Certainly there are pieces of data that are necessary for action, but I often find there are lots of additional fields that analysts want, but are more of a burden than a necessity.

]]>02:44cleanetlArchitecture for AuditingThu, 04 Jun 2015 01:00:00 +0000In the analog world, when we need to provide an auditing system for checking on physical activity, we usually separate out the roles of people that might record or track activity from those that perform activities. The better separation we have between individuals, the better we can usually ensure our auditing records are intact, complete, and trustworthy. We certainly have issues in the real world achieving these goals, but overall things work well.

In our digital systems, however, auditing seems to be bolted on to the systems and not designed with the need to separate duties and protect our auditing records from tampering. I would have hoped that many of the software platforms (Oses, RDBMSs, etc) over the years would have recognized this (along with security) and enabled auditing as a function separate from Administrator/root/sa/etc, but it hasn't happened. Far too often, we've operated under the assumption that administrators can, and should, be trusted.

]]>02:37cleansecurity,auditingFinding Production DriftTue, 02 Jun 2015 01:00:00 +0000A few years ago when I started delivering presentations about automating database delivery with Grant Fritchey, I was surprised by something. We were rehearsing a talk and Grant mentioned that there were really only two places where we knew the state of our code: in a VCS and in production. I hadn't really thought about that before, but realized it was true.

While the development environments are always in flux, by using a VCS, you can determine the state of your code at any point in time. However since changes can occur in production (DBAs adding indexes, changing security, etc), you can't count on a release branch of some other designation as being a guarantee of the state of your live environment. Production is always its own version of the truth.

]]>02:42cleanadministration,sql,serverYour JobMon, 01 Jun 2015 01:00:00 +0000This quote is a great: "I don’t really care if your code runs locally. If your code just runs locally, then my only option is to sell your computer so that our users can use our software, and that really doesn’t scale." That quote is from an open letter to engineers and talks about what the job of a software engineer really is. It's not just to write code, but to make sure our code and the entire system works for users.

That's important. It doesn't matter how well you code if the application only works on your machine. We traditionally haven't been great at getting code from one system to another and ensuring it works, at least not in a smooth fashion. We haven't really followed an "engineering" process when we write, and deploy, software.

]]>02:14cleansoftware,development,careerStatistical AnalysisFri, 29 May 2015 01:00:00 +0000I ran across a book recently that is written to address some of the statistical issues the author has found in many scientific papers. It's beenreleased online for free, but if you like it and would like a more portable format, you canbuy it for a reasonable price.

I haven't dug into the book much, and I certainly don't consider myself to be a statistician. That wasn't one of my stronger subjects in school, and it's a weakness I'd like to correct at some point. However I have had to use some of the statistical functions in past applications, and I wonder if I was using them wrong.

]]>02:15cleandata,anlysisThe Hidden Patterns in DataThu, 28 May 2015 01:00:00 +0000There's a great TED talk on statistics, and one I'd recommend you watch. However, it's not so much the topic and data presented, but the way in which it's presented and broken down that becomes interesting. If we ignore the topic, and just look at the analysis, a few things stand out that I'd apply to business.

The first noticeable thing for me was the breaking down of the large bubbles into smaller ones. I think that all too often we try to boil down information to a single, or very limited number, of values. While the overall average or graph of a set of data can be helpful, it can also be distorted by outliers in the underlying data. Seeing the data broken part into quintiles helps to analyze how closely the overall average matches up with the data.

]]>02:48cleandata,anlysisNative AuditsFri, 22 May 2015 01:00:00 +0000As our databases contain more and more information, it is becoming increasingly more common to implement some sort of auditing feature to ensure that our data is not being inappropriately accessed or altered. It's not only the management of our organizations, but also regulatory rules are becoming more numerous, and some may think onerous, which increases the burden on the DBA.

There are numerous products available to help here, but SQL Server has included it's own internal features since SQL Server 2008. However, it seems that I encounter many people that are unaware of the SQL Server Audit feature, and indeen, may not be aware of the Extended Events system on which it's based. This week I wanted to ask if you are using this feature.

]]>01:58cleansql,server,auditingNaming ConfusionThu, 21 May 2015 01:00:00 +0000When I first worked in a corporation that had many servers, I had to learn the encoding of our systems. We had SUIS01, SUIS02, NAIS01, NAOP01, etc. Each name designated a location and department, but no information as to what applications were running. That was a separate thing to learn, which thankfully didn't change often as procuring and setting up a new system was a relatively rare event.

However I also ran into a few systems named "Dorothy" and "Wiz". The other administrators griped constantly about the unprofessionalism of those names. I'm not sure I think SUIS01 is better or worse than Dorothy, but that's a separate debate. In both cases, we had to memorize what applications were actually running on which systems. Since all of them were managed by IS, regardless of the department that owned them, we were responsible for understanding the purpose of the server.

]]>02:43cleanadministrationControlling AlertsWed, 20 May 2015 01:00:00 +0000None of us likes to be on call for our organizations. Late night phone calls, especially for something silly like adding a new user to a system or rebooting a machine, can make for irritable employees in the morning. Too many nights in a row of problem calls, legitimate or not, and I'd argue that an employee's effectiveness during the day is substantially diminished.

Since many of us are paid to be creative and improve systems over time, is regular firefighting a good use of our knowledge? This might be even more true if we find ourselves receiving escalation calls on a regular basis and our daily workload doesn't change.

]]>02:47cleanadministration,sql,serverVMs are not VMsTue, 19 May 2015 01:00:00 +0000I was at VMWare recently. One of the main things that all of the SQL Server professionals that were there tried to emphasize is that SQL Server workloads are not like other workloads. The impact on the various host resources, the stress on the storage systems, these are fundamentally different in a database server. The loads tend to be higher, but not always, however the tolerance for delays tends to be lower than for many other types of applications.

This becomes an issue if you work in an organization that doesn't understand the challenges of database systems. It's entirely possible that your virtualization administators, or your storage administrators don't recognize that the SQL Server might need more resources. Or they don't believe the impact is greater for the organization. To be fair, that might be true, but someone other than the DBA or system administrator should decide if the database is more important than the file server and should be treated differently from an infrastructure perspective.

]]>03:00cleanvirtualizationBoring or ScriptingMon, 18 May 2015 01:00:00 +0000Do you want to continue to perform boring, repetitive, mundane tasks as a part of your job? Many of you might not be challenged at work, or you might be burdened with a series of requests that repeat themselves over and over. They're easy tasks, many of them probably take minutes. I'm sure there's also a level of mindlessness that you find comforting at times with just working through a familiar task.

However many of you also get busy. You have no shortage of new tasks that get assigned to you on a regular basis. You probably also get stressed from your heavy workload at times. What do you do when you're too busy to work through the mundane tasks, but they still need to be done? It can be a challenge to manage that burst in a workload if you haven't prepared for it.

]]>02:30cleancareer,powershellDocumenting with TestsThu, 14 May 2015 01:00:00 +0000Documenting code isn't a task that many people enjoy. Those of you that have been through a computer science curriculum probably added lots of verbose, and obvious, comments to code. It's probably unlikely that you ever found much value in the comments in a paying job.

I ran across an interesting approach from Ed Elliot on documenting your code with unit tests. I hadn't thought about the tests providing some documentation, but it's an interesting idea. I'd have to work with the concept a bit, but I'm skeptical I'd get enough information from unit tests to ensure I understood what a stored procedure was doing, especially if I had 10 tests for a long procedure.

]]>02:18cleansoftware,developmentOpening Up DataTue, 12 May 2015 01:00:00 +0000Tim O'Reilly has been an advocate of open data access and standards for some time, especially from governments. He's pushed for more interoperability and certainly more accessability from all sorts of groups. He gave an interview earlier this year to LinuxVoice where he talked about a variety of things, but data was foremost on his mind.

There are some good thoughts, but I was pleased to see him looking for more software to adapt how it works with data rather than asking data to match the application. An interesting thought he had was in the area of control systems. Does every device or sensor need a separate application and way of interacting or should we have some guiding design principles that let similar applications work in similar ways with different data? That almost sounds like good data modeling and normalization principles in action, backing a data driven application.

]]>03:27cleandataHow Virtualized?Fri, 08 May 2015 01:00:00 +0000I went to a talk recently where I saw this statistic: "50% of all workloads were virtualized in 2009. That number is 72% today."

That's a really big number, at least in my mind. That implies the vast majority of all servers, file, print, database, email, etc. are virtualized. Inside of companies that have their own data centers and machines, they must be heavily virtualized. I'm sure that all those instances in the "cloud" also count, but still, 72%? That's big.

]]>02:34cleansql,server,virtualizationvNext 2016Thu, 07 May 2015 01:00:00 +0000We have a name for the next version of SQL Server: SQL Server 2016. I suspect the internal build may be v14, which I guess makes sense if you're superstitious about the number 13, or you think your customers are. There are plenty of v13s out there for various other software packages, so I'm not sure if this is an issue, but we'll see when we get a public CTP for the next version of SQL Server.

vNext has been the way to refer to the next version of Microsoft products before a decision is announced about the official name. This convention seems to have taken over from code names, which makes sense, but it certainly is less fun than hearing Denali, Yukon, or some of the names that have been used for SQL Server.

]]>03:43cleansql,serverIdempotentWed, 06 May 2015 01:00:00 +0000I ran into the word idempotent in the Stairway to Integration Services. I had heard the word, but I hadn't really considered how important it can be for a DBA or developer until that time. It's a term used in computer science, as well as other sciences, but I think it's one that many of us don't consider when we're writing code, especially code used to deploy software to other systems.

]]>02:28cleansoftware,development,tsqlCan Auditing Fail?Tue, 05 May 2015 01:00:00 +0000I saw someone's list of requirements for a SQL Server auditing system recently that included many of the standard items most of us would want. DDL changes should be captured, along with configuration alterations. The performance impact should be low, and all versions of SQL Server must be supported. However, there was one requirement that surprised me. This person noted that the auditing must be asynchronous and not be dependent on the application using the database. The detailed description noted that if auditing failed, the application shouldn't have any of its transactions limited or rolled back.

I'm sure there are use cases where this is important, and where the auditing might not be critical, but the auditing data is available for informational purposes to troubleshoot any issues that relate to the database. However in many places where auditors review information, or the data is part of a legal record, the auditing cannot fail. If it does, then the application needs to stop working. If an audit is truly an audit of activity, then all activity must be audited.

]]>02:38cleansql,server,databases,auditingSource Code SecurityTue, 31 Mar 2015 01:00:00 +0000I'm not surprised, and I had expected to read about something like this much sooner. Apparently someone at the ride sharing company, Uber, posted a security key online in a GitHub distribution. I assume this was some sort of code repository for Uber that may or may not have been supposed to be shared publicly, but having used GitHub, I could see someone making a mistake and accidentally putting private code in the public space.

Uber is worried as the key is a security authorization key used to access their databases. Someone apparently downloaded Uber database files and now Uber is attempting to track them down. The whole situation is a mess, but there are a number of problems here that we could learn from.

]]>02:43cleansoftware,security,development,encryptionSoftware Engineering in PracticeMon, 30 Mar 2015 01:00:00 +0000I ran across a review of a developer's first year at a company. The company is endjin and the poster describes their first year as a "software engineer apprentice," which is an interesting way to look at one's early career. While it's not for everyone, I suspect that many developers in this business could actually use a bit of apprentice time to solidify their foundation of knowledge.

The post talks more about the way in which the work is done than the actual tools involved. A few are mentioned, but this is really a look back at the philosophical way in which the company solves problems. More importantly, it's the view of an apprentice learning a trade.

]]>02:09cleansoftware,development,engineeringTestingTue, 13 Jan 2015 02:00:00 +0000It seems that software always contains bugs. No matter how much time and effort is spent building an application, there will be issues. Sometimes this is because of a lack of testing, and sometimes this is because of poor testing, but in any case, the expectation that we will test our code is becoming more prevalent as we depend more and more on computer software. Users expect our software to work.

It seems there is never enough time to properly test software after it is complete. Perhaps your deadlines are too tight, perhaps there aren't enough resources to devote to comprehensive testing processes, but it really doesn't matter. We will never have enough resources in our QA and testing teams to do as much testing as we would like. We need to expect this and find ways to raise the quality of our work, given those constraints.

]]>02:12cleansoftware,development,testing,tsqltHoliday StrugglesTue, 09 Dec 2014 02:00:00 +0000It's the slow time of year. As I saw tweeted recently, "And now for the month when nothing gets done." I haven't usually felt this way at a few jobs as the budget and planning process has typically been a part of a busy month. As I've run SQLServerCentral, I've also been looking back and forward this month, trying to think about our industry and data in a long term way. I have found December to be frenetic as times as I try to get in some ski days while still getting work done.

However this year I've been in a situation that I think many people get into. I had a number of vacation days I needed to take, so I scheduled them a day here and there across the weeks. Between being off for Thanksgiving week, and missing almost the complete last two weeks of the year, I'm finding it a bit difficult to get into the swing of things as I have a few days back at work. It seems that work naturally slows down in teams as various people take holiday time off, but this is the first time that I've struggled to actually get my own work done as I'm in and out of the office on an irregular schedule.

]]>02:39cleancareerThe Real Scary DBAsMon, 08 Sep 2014 01:00:00 +0000I ran into someone recently that told me they were scared at their job. This person had built a number of ETL jobs to move data around between systems. They were trained as a developer, but had a little experience, got sucked into working with SQL Server, and had made a career without really considering themselves a DBA. They seemed a bit bewildered that they hadn't messed up the data in any of their employer's systems.

I've met quite a few people like that, who seem amazed to be trusted to accomplish the work they do on a regular basis. It's good that many people can go through a career in SQL Server and successfully accomplish the things their employer needs done. However there's no shortage of "scary DBAs" out there that are in over their heads and do cause damage to data and systems.

]]>02:24cleancareer,sqlserver,administratioData Decisions or Instinct?Thu, 24 Jul 2014 01:00:00 +0000Most of us that are data professionals think the best way to make decisions is to use data to justify some course of action. We look for patterns in data, some guidance that the information we have will lead us to make the best choice for our organization. Google has talked about making data driven decisions as a part of their success and they think more organizations should do this. Any number of other companies also use data to power their BI systems and dashboards that help their employees make better choices.

That seems in contrast to this piece from the Harvard Business Review that says that great decisions don't start with data. It talks about using stories and emotions, with a few key facts sprinkled in, to help sell ideas and get decisions made. On one hand I agree that stories help to sell decisions, but I often have found that successful salespeople use this technique to deceive and convince by plucking emotional heartstrings, and using relatively little data.

]]>02:40cleanscience,data,warehousingDev OpsTue, 22 Jul 2014 01:00:00 +0000The DevOps movement is supposed to promote a closer collaboration between developers, testers, operational people, and really anyone else that must help software gets build and deployed. The developers need to coordinate their needs and requirements with operational staff. They should work early on with any staff performing testing to help them ensure that bugs are caught quickly and fewer slip through to production. At the same time, operational staff should be responsive and helpful in all the environments software moves through. Operational staff should help build and automate the configuration of development and testing environments, ensuring that systems are built in a repeatable fashion as well as the same way in each stage of the software development pipline.]]>02:48cleandevops,softwaredevelopmentFarewellFri, 30 May 2014 01:00:00 +0000This is my last day of work this quarter as I start my sabbatical at 5 o'clock today. I'll be gone for the next six weeks, away from SQLServerCentral, trying to improve and grow myself while staying as unwired as I can during that time. The rest of the staff here at SQLServerCentral will pick up the load, and I'm sure everything will be fine.

This is a bit scary for me. I've pretty much been employed full time since I was 17, only taking a few, short, involuntary stretches of unemployment in almost 30 years. Even during those times without a job I was focused on finding a job almost every day, so it felt as if I still had the job of finding a job during those times. I'm nervous about taking the time off, and certainly worried a bit about SQLServerCentral as I'm not sure I've been away from the site for more than a week since it was founded.

]]>02:33cleancareerNo Works of ArtThu, 29 May 2014 01:00:00 +0000I used to pride myself on being able to run through menu selections and configuration options from memory. There have been times when I could describe over the phone the process for using a GUI. I've been able to direct people to log onto SQL Server, choose a database, right click it, and choose restore. Then select device, choose NORECOVERY, etc, etc. I could walk people through many Windows dialogs, IIS configurations, and more, with verbal commands, picturing what they would see on the screen as I was holding my phone to my ear.

People would be impressed, and I'd rarely make mistakes. I knew what was coming next, and where to click. However I did make mistakes at times, and while that's human, that's not what we want during software configurations. These days we find plenty of administrators and developers running an install program, and either selecting options from memory or using some sort of document that was given to them. The thing that concerns me is that any of those processes could be full of mistakes. Documents never get rev'ed to keep up with changes. Human memory is faulty, and certainly we find ourselves clicking the wrong button at times.

]]>03:26cleanadministration,software,developmentPatching ProblemsTue, 27 May 2014 01:00:00 +0000I applied the Windows 8.1 update a few weeks ago and had some issues with my machine. Windows was fine, but I lost my SQL Server service. A few other users, including some of the SQLServerCentral community also had issues and sent me ideas, but their fixes didn't work for me. That was OK because the problems gave me a chance to use PoSh to solve a real problem. I'll be blogging about that in the next week.

However the 8.1 update has caused lots of issues, and Microsoft is acknowledging these problems. That's good, but the process gives me pause, and to a large extent, I think this makes more and more people suspect about all of Microsoft's patching processes. I bet there are companies that feel even more justified in waiting for SP1 for SQL Server 2014 before upgrading, even though there is a chance that the patch itself will cause problems.

]]>02:57cleanmicrosoft,patching,sqlserverLaptop DreamsMon, 26 May 2014 01:00:00 +0000TheSurface 3was announced last week, and I was underwhelmed by the event, especially the presentation itself. The entire spectable didn't seem to compare to some other companies' announcements. I did, however, find myself thinking about replacing my laptop with a tablet that's lighter and allows me to forgo a keyboard. As much as I use a keyboard and depend on a responsive set of keys to get work done, there are definitely times when I'd like to just have a screen, and certainly a lighter form factor.

Today is Memorial Day in the US, and I'll be outside in a variety of situations today. Every year my son recognizes and remembers our soldiers at a Memorial Day ceremony with his fellow Boy Scouts. I like to capture some photos and video of the occasion, as well as track which Scouts are supposed to be there and have actually shown up. I can do it on a phone, but a laptop is too cumbersome. A tablet is actually the best option. It seems funny to use a larger device, but being able to capture and playback video on a tablet for others is much nicer than using a small phone.

]]>02:34cleanThe Development BackupTue, 20 May 2014 01:00:00 +0000Have you ever had a development server crash? Have you lost work because of this? Had delays or had to recreate code? You shouldn't, or at least you shouldn't lose much work or time..

There was a time when I offered to manage backups on all development servers. This was in a large environment with hundreds of instances. I wasn't worried. I had scripts to do the work of setting up, running, and reporting on backups for instances. I knew how to deploy these scripts to hundreds of servers.

]]>01:53cleanbackup,vcs,softwaredevelopment,sqlserverWhat are you worth?Mon, 19 May 2014 01:00:00 +0000Each of us is reponsible for negotiating his or her own salary for a position. Often we don't have much leverage to exact a higher salary, and even if it's deserved, so many companies don't have the flexibility for managers to pay higher salaries than what is set in some range by their HR group. That's discussed a bit in this post from Chris Shaw that looks at hiring and salaries.

I've always felt that the way we handle salary was developed by owners and managers, and the process benefits them, not the employees. We rarely discuss salaries at work, with disclosure sometimes forbidden. Your salary adjustments over time seem to be highly based on your current salary. That means that your starting salary often determines your future salaries at a company, regardless of your abilities relative to others or the market. Even changes in supply and demand may not affect your salary.

]]>02:35cleancareerFailureFri, 16 May 2014 01:00:00 +0000Tim O'Reilly has written some great pieces over the years on many aspects of technology and change. I've followed him for years and I've enjoyed many of his thoughts and views on our industry. I think he's thoughtful and truly envisions the amazing things that technology can do to change the world, mostly for the better. I think it's worth randomly reading through (or listening to) some of his thoughts.

He recently wrote a long piece on the failures he's had in business with his company,O'Reilly Media. It's an interesting tale that looks at process, at mis-communications, and business. I think about my career, looking back on past decisions, looking at the way in which companies I've been associated with are run, and at the way that others around me have worked. I realize that there are many failures to go with the successes in my life. The piece made me take a good, hard look at my past, drawing parallels to the ways in which leadership at O'Reilly has worked as well as how it has come up short.

]]>02:47cleancareerWhy Don't We Have Better Practices?Thu, 15 May 2014 01:00:00 +0000This essay isn't quite Safe For Work (SFW) and might offend a few people, so be warned, but I found it somewhat humorous. It's a look at all the insanity and problems with programming and software in the world. It certainly makes me glad that those of us working on software don't actually build tangible objects like bridges.

However the piece did make me think a bit. Why do we have so conflict and strange practices in many software systems? Why haven't we decided upon more good security practices that we widely disseminate? Why doesn't much of our code follow established patterns, like those that we might use to build a bridge?

]]>02:32cleansoftwaredevelopmentBetter TrainingTue, 13 May 2014 01:00:00 +0000I was talking with some data professionals recently about training and the value of college, work, or some other method of entering the technology business. I have a few thoughts about the different ways of teaching people about this business, and I'm curious what you think.

What do you think of traditional computer science? Colleges have students study computer science theories, languages, and more. Build small software projects, write pieces of operating systems and more as part of their curriculum. Students also still have the requirements of other core classes like science, language arts, etc., but does this train them well? I almost think that students ought to be charged with working on real projects, perhaps open source projects, and tackling some small part of the system. Perhaps an older student needs to do project management work, or architectural specifications for less experienced students that are actually programming. Is that good preparation for the real world?

]]>02:19cleancareerCalling Local SpeakersMon, 12 May 2014 01:00:00 +0000There's likely a call for speakers open somewhere near you. When I was early in my career, I'd see these calls at large conferences like Comdex, TechEd, even the PASS Summit, and I'd think I'd never get the chance to present, with the best of the best speakers being accepted and no chance that I would be chosen. Things have changed in the world with the growth of SQL Saturday, there are a dozen open calls right now, all around the world, with many opportunities for new people to give a presentation.

When Andy Warren came up with the idea for SQL Saturday, the idea was that we would mostly find local speakers to help present at the event. We thought it might be expensive, and hard, to get the same speakers that present at larger conferences. This event would give many of the talented DBAs in local areas the chance to speak to larger audiences than they get at many user groups. SQL Saturday has been so successful, and the growth is amazing. We've started to see more and more well known speakers traveling substantial distances to attend events. That's great, but I do worry about the lack of local speakers at some events I've attended.

]]>02:50nospeaking,career,presenting,sqlsaturdayBackup ResponsibilityFri, 09 May 2014 01:00:00 +0000One of the most important things that you can do as a DBA, or really as any sort of system administrator, is back up your system.Ensuring that you have have backups, and of course, that you can restore them, is the number one priority for sysadmins. Everything else that you need to do is second to backups. After all, backups ensure you still have a system after a disaster. If you can't do that, then security, performance, features, none of that matters.

I have worked in large and small environments, and in all cases, I've assumed that as the DBA, I need to be checking that backups are occurring, and that I can restore them in case of any issues. Often this has meant I need to work more closely with others that have the actual responsibility for performing backups. This week, I'm wondering how many of you work in similar situations.

]]>02:23cleanNo Handwaving Away the DBAThu, 08 May 2014 01:00:00 +0000There's a great quote I read, at the end of this article. It says: "...if you think that switching to NoSQL will just let you hand-wave away all of the challenges of running a database, you are terribly misguided." The context is that all too often people looking to move away from some of the hassles of working with RDBMS platforms, which includes working with the DBA, haven't completely thought through the issues.

I do think NoSQL has a place in the world. There are domains of problems that I'm sure Riak, MongoDB, and others, solve in a more efficient way than SQL Server, Oracle, MySQL, and other relational systems. I'm not sure what they are, and to some extent, I haven't seen good guidance on where particular platforms excel. Most of the articles and pieces on choosing NoSQL seem to be trying to sell me "why a particular platform can replace my other one", and telling me to add in things like transactions, but not explaining the drawbacks.

]]>02:49cleanadministration,databases,dba,nosql,sqlserverManagersWed, 07 May 2014 01:00:00 +0000Most of us have managers that are picked from the ranks of our technical staff. Often the most talented technical person, or most willing technical staff member, is promoted to manager, given a few hours of training on the various tools that he or she will use to deal with the administrative paperwork a manager fills out, and then cut loose to "manage" a group of people.

And they stink at it.

Most of them are bad. Not only are they bad managers, they are often bad leaders, and many find their technical skills suffer as they split their time with managerial tasks. That certainly puts them in a bad mood and creates worry about their future. It's not even just in the technology industry, as we find that many managers are bad, yet still languish on for years. This despite the fact that research shows they are costing their companies profit.

]]>02:10cleanVocation or CareerTue, 06 May 2014 01:00:00 +0000Career: an individual's journey through learning, work, and other aspects of life.

Vocation: an occupation to which a person is specially drawn or for which he or she is suited, trained, or qualified.

Which one describes your current employment situation? How do you feel about the technical work you do? A career is a good thing, and I hope that you all have long, successful careers, I hope you continue to learn and maintain a balance between work and your life outside of your job.

]]>01:54cleancareerThe Gadget ItchFri, 02 May 2014 01:00:00 +0000Many of the people that work in technology are interested in gadgets and hardware as much as software. Not all of us, but many of us have had side projects where we've melded hardware and software together. For those of us that are older, and started working with computers decades ago, that might have been the only way that we could get a computer system to work.

I'm glad that I still find people interested in hardware. My son recently built his own gaming computer from parts, and I'm hoping to distract him from some games and get him to help me build some robots with a Raspberry Pi computer and BrickPi add on board. However it's not just kids as I've seen a number of people at Red Gate working with hardware to build a variety of interesting projects.

]]>01:58cleansoftware,hacking,raspberrypiThe Subtle Push to the CloudTue, 29 Apr 2014 01:00:00 +0000With SQL Server 2014 released, there's the temptation to upgrade for many DBAs. However the licensing costs and debatable improvements in the product will temper the DBA's enthusiasm with the reality of the ROI seen by management. While reading about the licensing changes, I also saw this note from Tom LaRock, where he wrote about the features most of us aren't using. It made me think about upgrades, and perhaps the strategy Microsoft is employing.

As Tom mentioned, the features not being used are Enterprise Edition features. This prevents many of us from upgrading to use them because Enterprise Edition is so expensive. Actually, even Standard Edition is expansive these days, given the per-core licensing, and I suspect lots of companies with SQL Server 2008, 2005, even 2000 are debating whether or not the upgrade is really worth the cost.

]]>02:06cleancloud,microsoft,azure,sqlserverGood Practices for Software DevelopmentThu, 24 Apr 2014 01:00:00 +0000Would you post your password on a wall in your office? Of course not, because other employees, the cleaning crew, even guests walking around your office would be able to access your system with your account. When I read Brian Kelly's post on passwords in files, that's what I thought of. Sticking credentials in a file, where they're subject to any kind of search, is a bad idea.

However this happens all the time. Combine this with a few other "common practices" like using sa to connect to a database and building dynamic SQL, and you might as well just set blank passwords and invite someone to have fun with your database. It's sad that we continue to see these types of software development practices in 2014, and especially poor to see them from companies that sell software.

]]>02:08cleansoftwaredevelopment,sqlserverOpen Source the MCMWed, 23 Apr 2014 01:00:00 +0000It's over. There'sno more MCM programor certification from Microsoft. The last MCM test was given last year and no upgrades are planned for SQL Server 2012 or SQL Server 2014. That's a little sad, though there are quite a few of our SQL Server professionals that can still proudly wear the MCM title for the rest of their careers.

The MCM tests were designed differently than all the other MCP type tests, requiring more thought and deduction, as well as practical skills. The lab in particular was daunting to many of the MCMs, most of whom would tell you about the difficulties in getting through the scenarios in the limited time alloted. All of the people I've talked to found the challenge refreshing and also informative, enabling them to learn a few things about their knowledge, even from the problems they didn't complete.

]]>02:31cleancareer,certification,mcmDue DiligenceTue, 22 Apr 2014 01:00:00 +0000I often talk with people about building their brands and finding a way to ensure they are a highly desirable employee. One of the ways that I think people can do this is with a technical blog about their career. Having a technical blog allows someone to show off their skills in a particular area. The blog doesn't have to be ground breaking work or extremely innovative solutions to complex problems. While employers need those people, they also need people that do solid work every day on regular problems.

An interview isn't a great way to find good employees. Many of us have had experience with either (or both) side of the interview table and realize that interviews aren't necessarily that helpful. If we bothered to track the impressions we make of candidates and compare that to the actual work they accomplish over the first year or two, I suspect we'd find that we have no evidence that we were making great decisions. The success of employees seems to be a bit hit and miss and not based on our choices in interviews.

]]>02:33cleanblogging,careerConsolidation MattersMon, 21 Apr 2014 01:00:00 +0000Throughout my career I've been looking to consolidate SQL Servers when I find them. The typical employer I've had usually grows their IT infrastructure over time and many projects look like this:

Build or buy a software package

Buy a new server

Install SQL Server and one database for the application

Over time this means I find lots of individual servers running at much less than full capacity. That's something that DBAs like, because it means that we can handle the inevitable spikes in resource usage that our workloads will encounter. However that's not what the rest of the business, especially the financial management, wants. Underused resources mean money that isn't spent well.

]]>02:41cleanadministration,consolidation,sqlserverAdvice for NewcomersFri, 18 Apr 2014 01:00:00 +0000A friend recently was asked to give a presentation on their career to a group of 12 year olds. It was a challenge to engage the students, and my friend was surprised that very few of the kids were interested in technology. I was disappointed as well since I think this is a great career choice, and worth a little investigation, especially at that age.

It's hard to convince people to enter this business if they don't have any interest, but if they do, I'm wondering what you might tell them. Today I'd like you to think about your words of wisdom to someone interested in your job.

]]>01:57cleancareer,sqlserverOne Database to Rule Them AllThu, 17 Apr 2014 01:00:00 +0000This is what you build to juggle 6,000 tweets a second. That's the headline that caught my eye and it's about the challenges of Twitter and the data that they handle. Twitter definitely has a tough problem, one that few of us have, but perhaps they can help us learn to better deal with our own data from their experiences on an edge case.

The story is journalistic, not so technical, but it is interesting. Twitter has struggled with a blend of data that is partially crucial and must be consistent now (usernames) and other data that can be a bit out of date (tweets). They also have lots of unstructured data (photo/video) that is combined with more traditional, structured data. They've used a few different database platforms to store this data and assemble it with their application. That's the same things that most of us also do when we deal with many different types of data.

]]>02:02cleantwitter,databases,sqlserverHow Long Before You Upgrade?Fri, 11 Apr 2014 01:00:00 +0000It's 2014. SQL Server 2000 is 14 years old, but there are still quite of you managing instances. SQL Server 2005 is 9 years old, and I'm sure more of you still deal with that version. I know because I work for a software vendor and I'm constantly asked if our software will run against those two versions of SQL Server.

For many of you, however, if you're managing a SQL Server 2000 instance, it might only be 9 or 10 years old. Your company might still have been installing SQL Server 2000 in the year 2005. The same is true for SQL Server 2005. I wouldn't be surprised to find companies still installing 2005 instances in 2008 or even 2009.

Given the headlines, the news we find out about companies not disclosing security issues, the back doors and poor code in much software, is it any wonder that people think it's a "when" and not an "if"? Given the lack of realization from many companies that suffer incidents that they were even attacked, perhaps that's an assumption worth making.

Big Data is constantly in the news. We've been asked at SQLserverCentral to try and develop some articles, perhaps even a stairway to explain what Big Data is and how we might use it. I'm still trying to grasp the concepts myself, and unlike the amorphous cloud, I'm still looking for some good examples of what Big Data really is.

When I ran acrossthis piecewarning that Big Data isn't the final solution to all our questions in the world, I wasn't surprised. The piece notes thatGoogle Fluhasn't been very accurate in its predictions of outbreaks. At first glance, this gives lots of credence to the idea that the good, solid data analysis and mining techniques we've used for years are just as good as any new Big Data fad.

]]>01:54cleanbig,data,sql,serverLobbying for ChangeTue, 08 Apr 2014 20:03:26 +0000The Connect system from Microsoft doesn't work that well, but that doesn't mean we shouldn't continue to press Microsoft for change.