In the previous article, I showed how to take advantage of collations to add more unique and interesting names to databases. In that article, I alluded to the better collation to use but failed to be very explicit about it.

As Solomon Rutzky pleads in many articles, the best default collation to use is not any collation that starts with SQL, but rather the collation one should use is Latin1_General_100_CI_AS_SC. I agree. Let’s not use the old default collations anymore and start using more modern collations – even if Microsoft does not change the default, you should change your defaults in your builds to represent a better more modern collation!

Fun with Sessions

Let’s suppose I strongly suspect I am having an issue with the ghost cleanup process for a specific database called ‘👻’. I want to prove whether the suspicions are accurate or not so I resort to an old tried and true XE session to validate. Here is that session.

master.dbo.sysColumnst2--Lack of join criteria makes this a CROSS-JOIN

/* let's get our Blinky updated properly */

Updatehg

SetBlinky='GHOST ITEM '+CONVERT(VARCHAR(20),ISNULL(Pinky,0))

FromHalloween.Ghostshg;

--===== A table is not properly formed unless a Primary Key has been assigned

-- Takes about 1 second to execute.

ALTERTABLEHalloween.Ghosts

ADDPRIMARYKEYCLUSTERED(Pinky)

/* additional index to show the ghosts */

CreateNonClusteredIndexIX_GhostPinkyonHalloween.Ghosts (Pinky)

/* tweak the ghost cleanup with an undocumented TF

Required for DBCC Page results to show which pages have a GHOST entry

Don't try this at home - we are what you call experts (besides this is only a lab box!!) */

--DBCC TRACEON (661, -1)

--GO

/* Somebody made a mistake and thought Halloween was Oct 30 instead of Oct 31

we need to kill those records

*/

Deletetop(666)

FromHalloween.Ghosts

WHERESlimer='10/30/2014';

The best part is yet to come. With the event session running, I need to execute the preceding query. The query may take several minutes to complete – don’t worry yet. After the query completes, it is time to check the event session data to try and confirm the suspicions.

read xe data

Transact-SQL

1

2

3

4

5

6

7

8

9

10

11

12

SELECT

n.value('(value)[1]','nvarchar(60)')ASdatabase_name,

n.value('(@count)[1]','bigint')ASghost_count

FROM

(SELECTCAST(target_dataasXML)target_data

FROMsys.dm_xe_sessionsASs

JOINsys.dm_xe_session_targetst

ONs.address=t.event_session_address

WHEREs.name=N'👻'

ANDt.target_name='histogram')astab

CROSSAPPLYtarget_data.nodes('HistogramTarget/Slot')asq(n)

;

This is where things start to get a little interesting! If I try to open the session data to view it from the GUI, I will see something like the following.

Well, that is not very useful! Let’s try it from a TSQL query instead!

Well, that seems to be a little better. At least I get some data back at this point. Sadly, the database name is poorly displayed. That said, the database name that is displayed is accurate – just not very useful. As it turns out, XE is lagging a bit behind in the ability to display the characters that require certain code points (seemingly any that require multiple code points or above a certain range). I have not yet tested the full limitation, just understand that there is a limitation.

The event session will start successfully. In addition, I can confirm that the event file is created on disk.

Then I will rerun the experiment to test the ghost_cleanup process against the 💩 database. If I query the event session with TSQL, I will see something like this.

If I look at the event session from within the GUI, I will see something slightly more usable.

Peeking at the file data (instead of displaying it in tabular format) from tsql commands, I will see something like this.

Stay tuned to see how we can resolve this issue where the 💩 and 👻 are not being translated properly in these TSQL queries for these sessions.

Put a bow on it

A picture is worth a thousand words, right? Sometimes, a picture for an event session just may be able to say it better than 50-60 letters used to try and describe the session. Maybe you just want to try it out for fun. Either way, the use of these characters in an event session can be a little difficult when looking to review the data.

Interested in learning about some deep technical information instead? Check these out!

Yes Virginia, there are ghosts in your database. More specifically, there are ghosts in your SQL Server database. They are not there to haunt you. They are not there just for this holiday season (speaking of Halloween Month).

How can there be ghosts in the database?

Why would there be ghosts in the database?

Do they happen because somebody issued a KILL statement?

Let’s address each of those in turn. A database ghost record is (in a very basic form) one that’s just been deleted in an index on a table . Delete operations don’t actually physically remove records from pages – they only mark them as having been deleted (ghosted). Now why is it done this way? The answer here is largely performance based. This is a performance optimization that allows delete operations to complete more quickly. Additionally, it allows the rollback of delete operations to process more quickly. The rollback processes faster because all that needs to happen is to “flip the flag” for the records as being deleted/ghosted, instead of having to reinsert the deleted records. That may be a bit over-generalized, but I hope you get the gist. In short, records are marked as “ghosted” when a delete operation is performed; and to rollback, you simply undo that mark.

Now, what about this KILL statement thing? The kill statement is pure Halloween fun and does not create ghost records.

Ghost Hunting

Now that we have established the purpose of Ghosts in the database, how do you verify the existence of Ghosts? In other words, what can we do to prove there really are spectral things in the database? This is where the fun really begins. First, we need to get out the equipment and tools (as any good ghost hunter would do) so we can capture these phantasms. Let’s call the first tool the “trap”. Here is what you will need for it.

Master.dbo.SysColumnst2--Lack of join criteria makes this a CROSS-JOIN

/* let's get our Blinky updated properly */

Updatehg

SetBlinky='GHOST ITEM '+CONVERT(VARCHAR(20),ISNULL(Pinky,0))

FromHalloween.Ghostshg;

--===== A table is not properly formed unless a Primary Key has been assigned

-- Takes about 1 second to execute.

ALTERTABLEHalloween.Ghosts

ADDPRIMARYKEYCLUSTERED(Pinky)

/* Make sure the soylent green blob is BIG */

--UPDATE Halloween.Ghosts

-- SET TheBlob = TheBlob + REPLICATE('Casper is not a real Poltergeist! Frankenstein hates being called Frankenmeanie.',15000);

--GO

/* additional index to show the ghosts */

CreateNonClusteredIndexIX_GhostPinkyonHalloween.Ghosts (Pinky)

This trap, err database, can be a bit large. As currently configured, we will need about 16GB of disk space to support it. If that is too much, I recommend removing the last column – “TheBlob”. As you can see, we are setting a rather large trap. The table we create (Halloween.Ghosts) will receive One Million records. This is most probably overkill to catch these ghosts, so you can also cut back on the number of records to be affected.

Now, to make sure we have some data and that we can use the table, let’s just run a little test query.

Transact-SQL

1

2

Selecttop100*

FromHalloween.Ghosts

Excellent, we have a good sample of data.

At this point, it is important to note that we have done nothing that will cause database ghost records. All that has been done is to set the framework so we can see the ghosts. With the framework in place, let’s try to catch some ghosts. To do so, we need to try to delete something. Since we just happen to have had a clerical error in our database, we have 666 prime candidates to try and fix. We happen to have several records that were supposed to be given a Slimer date of Halloween. The clerk, being absent minded, thought that Halloween was supposed to be on Oct. 30. Our business model dictates that the invalid records must be deleted first and then we can try to enter the replacement records. So, let’s go ahead and try to remove those records.

Before we remove the records though, we need to discuss one important requirement for us to be able to see the ghosts. Let’s call it spectral vision goggles. In the database realm, we call it a trace flag. In order to see the the ghosts on the pages, we need to enable TF 661. We can do that with the following statement. There is a serious side effect to this method too – it alters the behavior of the Ecto Containment Unit or automatic ghost cleanup process. If you enable this, you will need to disable it later and/or manually run a ghost cleanup.

Transact-SQL

1

2

3

4

5

/* tweak the ghost cleanup with an undocumented TF

Required for DBCC Page results to show which pages have a GHOST entry

Don't try this at home - we are what you call experts (besides this is only a lab box!!) */

DBCCTRACEON (661,-1)

GO

Now that we have the last piece of equipment in place, let’s go ahead and try to delete some records.

Transact-SQL

1

2

3

Deletetop(666)

FromHalloween.Ghosts

WHERESlimer='10/30/2014';

With all of those records deleted (all 666 of them), let’s see what we might have captured. First, let’s take a look at some index stats.

If we look at the output of this query, we will see that we did indeed attempt to delete 666 records. Those records will now display in the ghost_record_count column. We will also see that, since we had two indexes on the table, there are 666 ghost records marked on each index.

Very cool! We are definitely on the track to capturing those ghosts. We have a trail that they exist in the database. Let’s keep going and see where we can see them. You should note that there is an additional column in our result set that looks like it might be related to ghost records. We are going to leave the discovery of version_ghost_record_count as a homework experiment for you to perform. It is beyond the current scope of this article.

Now this is getting exciting. We have stronger evidence in the log showing that these ghosts are hanging around in the database. Not only are they hanging around in the database, we can see which pages in the database on which they are trying to hide.

This is really solid information! fn_dblog is giving us just about everything we need in order to get those ghosts. It took a little bit of work since the log reports the page number in hex. Converting that to an integer page number is essential for us to look at the page (besides integer values are easier to interpret for most people). Now I can take that PageID and pass that number, for any of the records reported by fn_dblog, and pass it into yet another undocumented procedure known as DBCC Page.

When looking to use DBCC page, we can either look at the PFS Page and see more pages that have ghost record counts. Or we can take the results seen from the fn_dblog output and then look at the contents of the page and catch those ghosts. We will take a quick look at the PFS page first. Then we will take a look at an index page next. In this database that we have created, the PFS page will show several other pages that have ghost records on them. Due to the size (over 2 million pages), we only see index pages with ghost records in that result. If our database were smaller, we would quite possibly see data pages in our first PFS page of the database. Let’s see a sample from the first PFS in this database.

Transact-SQL

1

2

DBCCPAGE('Sandbox',1,1,3)WITHTABLERESULTS

GO

We can follow that link from this point to page 126. Page 126 happens to be an index page similar to the following. There are a couple of indicators that this is an index page. First being that when we run DBCC Page with a format of 3, we will see two result sets. The second result set will show statistics and index information. The second being in the image attached after the query. We will leave it as an exercise to you to see other ways to demonstrate that this is an index page.

Transact-SQL

1

2

DBCCPAGE('Sandbox',1,126,3)WITHTABLERESULTS

GO

That is great, but we have more ghosts to find. Let’s look at a ghost on a data page. Randomly picking a PageID from that list that was output from fn_dblog, let’s see what DBCC Page will provide to us.

Transact-SQL

1

2

DBCCPAGE('Sandbox',1,1522936,3)WITHTABLERESULTS

GO

Conclusion

Well, isn’t that just cool! We have trapped a bunch of ghosts and were even able to see them. This has been a fantastic deep dive into the crypts of the database. This is merely a scratch on the surface though. We hope this will encourage you to explore a bit and at least try one of the homework assignments we left behind in this article.

With all of that, we have a bit of cleanup to do. The cleanup comes in one of two methods. Method one involves manual labor. Method two involves our friendly little trace flag we already used. Since most DBAs prefer the automated mechanisms over manual, let’s just discuss method two for now. It is extremely effortless.

Transact-SQL

1

2

DBCCTRACEOFF (661,-1)

GO

That will put the system back to the way it was when we started (and of course we trust that nobody did this on their prod box).

This has been one of a few articles about ghosts in the database. You can check out some of the others here and here.

Halloween is a great time of year. It is unfortunate that it is just one day of the year. That said, I do like to think of the month of October as Halloween Month. I have several posts over the years that geek out over the cross-over between Halloween and SQL Server.

The undead of Halloween are now upon us. Among the hordes of data zombies roaming the streets in lab-coats and fishnet stockings, few of us are still scrambling to remove the hexes we have looming over our data.

As chance would have it, these hexidecimals, err hexes, have a more profound effect on us than we first thought. Many may have yet to even recognize the impact of the hexes placed along with the monsters that now lurk in the data after having been summoned via those hexes.

DB and Fun Related

Seeing as I am a really big fan of this holiday I have a few Halloween posts over the years. If you are interested in the previous Halloween posts, here is a list of a few of them:

That list is my Halloween treat this year. Now for a bit of a trick with a very strong warning. Because of this warning, I am not posting any code showing how to perform the trick.

Warning

The contents of this post are for the intent of HUMOR!

Freddy Kreuger

I can hear you clamoring from half a world away right now – “There is nothing called ‘Freddy Kreuger’ in SQL Server. What in the world are you talking about?”

You would be very accurate in your exclamation there. But this is not necessarily a strict exercise in feature names within SQL Server. I want you to think a little further outside the norms for a while.

Do you currently or have you ever needed to shred XML? XML shredding via TSQL can be a monstrously bloody killer to your database performance. As it turns out, Mr. Kreuger was also a monstrously bloody shredder.

Jason Voorheese

Yet another beast that is not truly in SQL Server, or is it? A not so new but new feature in SQL Server is called JSON. This feature does actually perform better than XML in some regards. That said, we do have a very common problem between the two of these features – blobs.

If you are not familiar with what that means -here you go. A blob is an overly large item being stored in the database. If you wish, you could correlate that to the other well known Halloween beast – “The Blob”.

Over time, this blob acts like sludge and just slows down your database queries. In addition, like the creature, the blob in your database tends to continue to grow in size and is seemingly never able to be put in check.

Skeletons

When I find skeletons, I have to be honest, I don’t find them terribly frightening. When talking about skeletons in your database, I am even less frightened.

Then again, when I run into the situation as described recently, in this post, I may get a bit of a startle and get just a wee bit concerned.

Overall though, I am rarely startled or frightened by any skeletons in the database. These are really just the supporting structures of a nice secure database and are called “schemas”. See, not really all that frightening here if we think about it just a bit.

This next one however, might be a little harder and should be nearly enough to cause some heart pain.

Warlocks

Surely there are no wizarding type of people in the database, right? Warlocks? I know for absolute certainty that there is no such feature or anything remotely close to a warlock, witch or wizard within the database. That is unless my database is about mystical creatures and people.

Alas, I urge you again to expand the box of perception a little bit and become just a tiny bit imaginative. This one, truth be told, does require a fair amount of explanation and imagination though.

The problem comes in part from some magical data issues that can occur due to this particular feature. In addition, this also comes from the wonderful grammatical errors from various blog posts and forums out there mis-spelling “which” as “witch”. Since “manwich” is really close to “man-witch”, I am calling it a warlock.

Now, since I am calling it a warlock, that leads us to the next strong hint about the feature. “Lock” in this case is the key. Now which magical, imaginative feature might there be that is related to “lock”? That would be the “nolock” directive and all of the data quality issues that it presents. Here is a really really good recap (by Aaron Bertrand) on this feature along with reference to it being “magic” – at this site.

Pirates

This is probably the easiest of the day by far. For all the data loving geeks out there, SQL Server has this pirate flavored way for you to get your drool on. This feature is called “R”. Yup – just like what a pirate says matey.

R is a tool to be used by data scientists or data geeks in general to try and throw together many different flavors of statistical analysis about your data.

Split Brain

Finally, (at least for this Halloween) we have this condition that is real within SQL Server. While treatable with long hours and heavy medication, it is something to be feared.

This condition is something rare but it is very real. The split brain syndrome is pretty much a multiple identity personality disorder in your database. If you have multiple nodes in a cluster, mirror or availability group, it is possible for more than one of those nodes to believe it is the master node and then for different transactions to become hardened in each of those nodes.

When this happens, you will not be able to use bleach to clean up the mess. Instead, you will be required to spend a grundle of time with your database cuddling it and nursing it back to data consistency and good mental health.

Seattle SQL Pro Workshop 2017

October is a great time of year for the SQL Server and Data professional. There are several conferences but the biggest happens to be in the Emerald City – Seattle.

Some friends and I have come together the past few years to put on an extra day of learning leading up to this massive conference. We call it the Seattle SQL Pro Workshop. I have created an Eventbrite page with all the gory details here.

That massive conference I have mentioned – you might have heard of it as well. It is called PASS Summit and you can find out a wealth of info from the website. Granted there are plenty of paid precon events sanctioned by PASS, we by no means are competing against them. We are trying to supplement the training and offer an extra avenue to any who could not attend the paid precons or who may be in town for only part of the day on Tuesday.

This year, we have a collision of sorts with this event. We are holding the event on Halloween – Oct 31, 2017. With it being Halloween, we welcome any who wish to attend the workshop in FULL costume.

So, what kinds of things will we cover at the event? I am glad you asked. Jimmy May will be there to talk about IO. Gail Shaw will be talking about the Query Optimizer (QO). Louis (Dr. SQL) will be taking us deep into Hierarchies. Andy Leonard will be exploring BIML and Wayne Sheffield will be showing us some SQL Injection attacks.

That is the 35,000 foot view of the sessions. You can read more about them from the EventBrite listing – HERE. What I do not yet have up on the is what I will be discussing.

My topic for the workshop will be hopefully something as useful and informative as the cool stuff everybody else is putting together. I will be sharing some insights about a tool from our friends over at Red-Gate that can help to change the face of the landscape in your development environments. This tool as illustrated so nicely by my Trojan Sheep, is called SQL Clone.

I will demonstrate the use of this tool to reduce the storage footprint required in Dev, Test, Stage, QA, UAT, etc etc etc. Based on client case study involving a 2TB database, we will see how this tool can help shrink that footprint to just under 2% – give or take. I will share some discoveries I met along the way and I even hope to show some internals from the SQL Server perspective when using this technology (can somebody say Extended Events to the Rescue?).

Why Attend?

Beyond getting some first rate training from some really awesome community driven types of data professionals, this is a prime opportunity to network with the same top notch individuals. These people are more than MVPs. They are truly technical giants in the data community.

This event gives you an opportunity to learn great stuff while at the same time you will have the chance to network on a more personal level with many peers and professionals. You will also have the opportunity to possibly solve some of your toughest work or career related problems. Believe me, the day spent with this group will be well worth your time and money!

Halloween is a great time of year. It is unfortunate that it is just one day of the year. That said, I do like to think of the month of October as Halloween Month. I have several posts over the years that geek out over the cross-over between Halloween and SQL Server.

With the proximity to Halloween, it’s not hard (knowing me) to figure out why I originally decided to terminate this series on the Eve of Halloween day. Then again, as it turns out, I decided to bring the series back to life on the other side of Halloween. In short, it was to be killed on Halloween Eve and then implicitly converted to some living dead creature. You pick whether it is a zombie (which is far better suited to SQL Server) or a vampire.

If you are interested in the previous Halloween posts, here is a list of a few of them:

XE Related

DB and Fun Related

That list is my Halloween treat this year. Now for the trick with a very strong warning. Because of this warning, I am not posting any code showing how to perform the trick.

Warning

DO NOT ATTEMPT THIS ON ANY PRODUCTION SERVER. I BEAR NO RESPONSIBILITY FOR ANY PRODUCTION SERVER CHANGES DUE TO THE READING OF THIS ARTICLE (OR ANY OTHER ARTICLE I HAVE WRITTEN). SOLE RESPONSIBILITY OF CHANGES OR ACTIONS TAKEN BELONG TO THE PERSON PERFORMING THE ACT OR CHANGE. IN OTHER WORDS, YOU ARE SOLELY RESPONSIBLE FOR BREAKING YOUR PRODUCTION ENVIRONMENT IF YOU IMPLEMENT THIS IN PRODUCTION!!!

I have debated seriously over even publishing this “trick” for Halloween because of how deliciously evil it is. I will try and paint the picture in broad strokes. I will leave it up to you to connect the dots.

Problem

A third party vendor has provided an application along with some database requirements. Among these requirements is that the application account must use the ‘sa’ login. You have a strict policy that ‘sa’ must never be used for any applications or by the end-users. This is an extremely protected account by policy. The dilemma you have been presented is that the CEO insists that this application must be used (never happens right?) and the vendor insists the application will not function properly without the use of ‘sa’ (I can hear you chortle at that).

Now you, as the DBA, are stuck between a rock and an even harder place. Being smart (and somewhat smart and sadistic), you insist that the use of the ‘sa’ login should not be performed because it will break SQL Server (in reality we know the login does not break SQL Server, but something inevitably will break due to a mis-step by the person using it a) when they shouldn’t, and b) because they lack proper training). Inside you are grinning from ear to ear because you have some devilish ideas, some of which you learned by attending a session by Rob Volk (twitter) where he shows some devilish methods to deter the use of ‘sa’.

For the sake of the scenario, let’s just say you have a policy preventing the implementation of logon triggers (as suggested by Rob) due to a couple of mishaps a few months back. Somebody implemented a logon trigger that wasn’t properly configured and it happened to prevent all users from accessing the server (including the dba’s). Ruh roh!

And then…

Later in the week, after reading about the power of Extended Events, you decide to play around and do a little discovery. You come across this blog post that shows you how to find all of the possible events within the XEvent Engine. So you run the script that you found and shown here:

event query

Transact-SQL

1

2

3

4

5

6

SELECTxo.nameASEventName,xo.description

FROMsys.dm_xe_objectsxo

WHERExo.object_type='event'

AND(xo.capabilities_desc<>'private'

ORxo.capabilities_descISNULL)

ORDERBYxo.name;

And while looking through the events and descriptions you discover that there is an event to log all of the successful logins. Not thinking anything of the third party vendor (because it just makes your blood boil) you begin to dream of the potential for auditing all successful logins (established connections) for all users and documenting who may be doing what and when.

After taking a few notes about the potential for the login event and getting access and logins under better control, you continue along your journey through Extended Events by reading this article about Actions. Feeling comfortable enough from the article, you do what any good data professional, trying to learn a new topic, would do – you start exploring using the provided query:

action query

Transact-SQL

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

SELECTxp.nameASPackageName

,xo.nameASActionName

,xo.descriptionASObjDescription

,xo.capabilities_desc

FROMsys.dm_xe_packagesASxp

INNERJOINsys.dm_xe_objectsASxo

ONxp.guid=xo.package_guid

WHERE(xp.capabilitiesISNULL

ORxp.capabilities&1=0

)

AND(xo.capabilitiesISNULL

ORxo.capabilities&1=0

)

ANDxo.object_type='action'

ORDERBYActionName,PackageName;

While looking through the available actions, you see one in particular that causes you to raise an eyebrow (not both, just one). There is an action called “debug_break”. You squirm and ponder for quite a bit at the name and definition of this particular object. You wonder out loud “why would anybody ever want that in there?”

Your co-worker interrupts with “Do what?”

To which you reply “Oh nothing important. It was just something I read.” After which you dismiss it, realize the time of day, pack up and head home for the evening. Hours later after the work day has long since been finished, you awaken in a cold sweat with an “Aha!” that startles your slumbering neighbors dog. Immediately you pull out your laptop, connect to your local sandbox instance and get to work with a wry smile and devious giggle.

Upon returning to work the next day, you call the vendor and work out a solution to build them a sandbox server to provide a proof of concept. You grant them the use of sa to use for their application login. Sitting down together and connected to the new sandbox, the vendor attempts to login and the following happens:

Slack-jawed and speechless the vendor pales in complexion. Deep down inside you giggle like an elementary grade school girl – content with yourself. BATTLE WON!

After the vendor departs with a promise to rework the security requirements, you restart the service on the sandbox and go about the rest of your work week with a silly grin from ear to ear and nobody the wiser.

That concludes the trick. In the tradition of Halloween, you knocked on my virtual door of Extended Events and I, in accordance with the traditions of Halloween, gave you a trick and a treat.

I hope this helps you to understand the power of Extended Events. Thinking through a problem and reaching a solution is what XEvents can help to provide. I hope this Halloween XE article was enjoyable to you. Stay tuned as the XE 60 Day series goes full Zombie starting next week.

This has been another article in the 60 Days of XE series. If you have missed any of the articles, or just want a refresher, check out the TOC.

This post will serve as the landing page for a series I am calling 60 Days of Extended Events. The purpose of these posts will be to help somebody progress through the Extended Events Concepts and become more proficient at implementing and using them.

The 60 days of XE will be a two month series running from September 1, 2015 through October 30, 2015. The posts will go live only on weekdays. So while it is 60 calendar days, it will end up being somewhere in the neighborhood of 45 articles.

These are the types of articles designed to take 5 to 15 minutes to learn a concept and move on with the work day. The articles are designed to build on previous concepts.

As the posts go live, I will return and update this page with the article title and link.

At the end of this two month series, there is still just too much to cover. Since I have plenty more in my queue, I have decided to extend the series into overtime. I have no guarantees on how many more posts I will add to the series from here, but I will continue to add more into the series until the queue is depleted.

This is the last article in a mini-series diving into the existence of ghosts and how to find them within your database.

So far this has been a fun and rewarding dive into Elysium to see and chat with these entities. We have unearthed some means to be able to see these things manifesting themselves in the previous articles. You can take a look at the previous articles here.

For this article, I had planned to discuss another undocumented method to look into the ghost records and their existence based on what was said on an msdn blog. But after a lot of research, testing and finally reaching out to Paul Randal, I determined that won’t work. So that idea was flushed all the way to Tartarus.

Let it be made very clear that DBTABLE does not offer a means to see the ghosts. Paul and I agree that the other article that mentioned DBTABLE really should have been referring to DBCC Page instead.

Despite flushing the idea to Tartarus, it was not a fruitless dive. It just was meaningless for the purpose of showing ghosts via that DBCC command. I still gained value from the dive!!

All of that said, the remainder of the plan still applies and it should be fun.

Really, at this point what is there that hasn’t been done about the ghosts? Well, if you are well tuned to these apparitions, you may have received the urge to explore them with Extended Events – sometimes called XE for short.

As has been done in the past, before we board Charon’s boat to cross the River Styx to Hades to find these ghosts in Elysium, one really needs to run the setup outlined here.

With the framework in place, you are now ready to explore with XE.

Transact-SQL

1

2

3

4

5

6

7

8

9

10

SELECTc.object_nameasEventName,o.descriptionasEventDescription

FROMsys.dm_xe_objectso

INNERJOINsys.dm_xe_object_columnsc

ONo.name=c.object_name

ando.package_guid=c.object_package_guid

WHEREobject_type='event'

ANDc.name='channel'

AND(o.descriptionlike'%ghost%'

ORc.object_namelike'%ghost%')

OrderByo.package_guid;

Look at that! There are several possible events that could help us track these ghosts. Or at the least we could get to know how these ghosts are handled deep down in the confines of Hades, err I mean the database engine.

From these possible events, I opted to work with ghost_cleanup and ghost_cleanup_task_process_pages_for_db_packet. The sessions I defined to trap our ghost tracks are as follows.

You can see there are two sessions defined for this trip down the Styx. Each session aptly named for our journey. The first (GhostHunt) is defined to trap ghost_cleanup and sends that information to a histogram target. The second (SoulSearch) is defined to use the other event, and is configured to send to the ring_buffer. Since the second event has a “count” field defined as a part of the event, it will work fine to just send it to the ring buffer for later evaluation.

Once I have the traps, I mean event sessions defined, I can now resume the test harness from the delete step as was previously done in previous articles. The following Delete is what I will use.

Transact-SQL

1

2

3

4

5

6

7

/* Somebody made a mistake and thought Halloween was Oct 30 instead of Oct 31

we need to kill those records

*/

Deletetop(666)

FromHalloween.Ghosts

WHERESlimer='10/30/2014';

Prior to running that delete though, I checked the Event Session data to confirm a starting baseline. Prior to the delete, I had the following in my histogram target.

After running the delete, and checking my histogram again, I see the following results.

You can see from this that in addition to the 25 pre-existing ghosts, we had another 672 ghosts (666 of which were from the delete).

This is how I was able to investigate the GhostHunt Extended Event Histogram.

Transact-SQL

1

2

3

4

5

6

7

8

9

10

11

12

13

/* Query the histogram */

SELECT

n.value('(value)[1]','nvarchar(60)')ASdatabase_name,

n.value('(@count)[1]','bigint')ASghost_count

FROM

(SELECTCAST(target_dataasXML)target_data

FROMsys.dm_xe_sessionsASs

JOINsys.dm_xe_session_targetst

ONs.address=t.event_session_address

WHEREs.name='GhostHunt'

ANDt.target_name='histogram')astab

CROSSAPPLYtarget_data.nodes('HistogramTarget/Slot')asq(n)

;

But what about looking at the other event session?

Let’s look at how we can go and investigate that session first and then look at some of the output data.

Cool! Querying the SoulSearch session has produced some information for various ghosts in the database. Unlike the histogram session that shows how many ghosts have been cleaned, this session shows us some page ids that could contain some ghosts – in the present. I can take page 1030111 for instance and examine the page with DBCC PAGE as follows.

Transact-SQL

1

DBCCPAGE('cemetery',1,1030111,3)withTABLERESULTS;

Look at that page and result!! We have found yet another poltergeist.

RIP

Once again we have been able to journey to the depths of the Database engine and explore the ghosts that might be there. This just happens to illustrate a possible means to investigate those ghosts. That said, I would not necessarily run these types of event sessions on a persistent basis. I would only run these sessions if there seems to be an issue with the Ghost cleanup or if you have a strong penchant to learn (on a sandbox server).

Some good information can be learned. It can also give a little insight into how much data is being deleted on a routine basis from your database. As a stretch, you could even possibly use something like this to get a handle on knowing the data you support. Just be cautious with the configuration of the XE and understand that there could be a negative impact on a very busy server. And certainly proceed at your own risk.

Now that you have read that other article, we can recap it a little bit.

You will recall that in that article, I discussed the presence of Ghosts in your database. I also discussed that those ghosts are a good thing. They have some benefits to them, of which is a bit of a performance boost for some operations like the rollback of a delete.

In that article I discussed one method with which you could see these ghost records. In this article, I would like to share an alternate method to be able to see these ghosts.

In order to explore this alternate means, let’s go ahead and follow the first few steps from the other article to get the setup complete so our second ghost hunting foray can begin.

Master.dbo.SysColumnst2--Lack of join criteria makes this a CROSS-JOIN

/* let's get our Blinky updated properly */

Updatehg

SetBlinky='GHOST ITEM '+CONVERT(VARCHAR(20),ISNULL(Pinky,0))

FromHalloween.Ghostshg;

--===== A table is not properly formed unless a Primary Key has been assigned

-- Takes about 1 second to execute.

ALTERTABLEHalloween.Ghosts

ADDPRIMARYKEYCLUSTERED(Pinky)

/* additional index to show the ghosts */

CreateNonClusteredIndexIX_GhostPinkyonHalloween.Ghosts (Pinky)

Now with the setup complete, we should once again confirm that we have appropriate data available for the hunt. Once again a simple query can suffice to show the data.

Transact-SQL

1

2

3

/* Take a peek at the data */

Selecttop100*

FromHalloween.Ghosts

Upon execution of the check script, we should see something similar to the following data-set.

Great, the data is confirmed and we should have a wonderful chance to find some ghosts once again. In the previous attempt, we needed an additional plasma blaster in the form of a trace flag. In this attempt we will hunt these ghosts without that tool and see what we can find. Why attempt it in this way? Well, it is simply because I’d rather not use a trace flag if it is not necessary. If you recall, that trace flag had a couple of noted effects. One of the effects was that it turned off the ghost cleanup process. If I can avoid it, I’d rather leave the containment unit in tact.

Now, due to that clerical error of putting Halloween on the wrong date, we need to follow the prescribed policy to delete records prior to creating the proper records.

Transact-SQL

1

2

3

4

5

6

7

/* Somebody made a mistake and thought halloween was Oct 30 instead of Oct 31

we need to kill those records

*/

Begintran

Deletetop(666)

FromHalloween.Ghosts

WHERESlimer='10/30/2014';

Notice that we issued the delete in a transaction this time around. We are going to leave that transaction open for a bit while we hunt those ghosts.

The first step is to verify that some ghosts might be present. To do that we should run a query such as the following.

With that query, we should see something like the following result set.

Very cool. We once again can see that these phantasms are in the database. We have enough information that we can proceed on to the next stage. We can pass the PageID into DBCC PAGE in order to investigate the ghosts on the page. If we use the PageID that is circled with the green in the preceding result set, we can get a better feel for these specters.

Transact-SQL

1

2

DBCCPAGE('Cemetery',1,2002440,3)WITHTABLERESULTS

GO

And we may see results such as the following.

Recall that the log says this page has ghost records on it. When we check the page with DBCC PAGE we can definitely see that there are ghosts on the page. This is very cool. Now, had we tried to check for ghost records on the PFS page we would not be able to see the ghost count like we were able to see by enabling the TF.

Once again we have been able to display the existence of ghosts in the database. In order to get these ghosts to move on from the database to their afterlife, we merely need to commit the transaction or roll back the transaction.

Stay tuned for the next article in this mini-series about ghosts. Who knows, we may even have a bonus Halloween article since this is Halloween month.

Something I have a hard time by-passing is a good R2 unit. I have R2 units in so many different forms, including a standing R2 cake one year for my birthday. So when I cam across this R2 unit, I just had to share it.

That is a pumpkin carved into the resemblance of R2-D2. I think it is a mighty fine job too. It’s amazing how many good Star Wars related pumpkin carvings there are out there. You probably wouldn’t have too difficult a time finding three or four hundred if you tried a little google-fu.

Each year I try to have something for the Halloween Holiday such as this one or this one. I failed to provide something in 2012, and this is getting back on the right track.

Despite the ease to find haunting Halloween effects related to SQL Server, I am amazed at how few have even heard of “Halloween Protection” which stems from the “Halloween Problem.”

I am not going to dive into the problem or the protection of it. I think that has been covered plenty and even quite masterfully by Paul White (blog | twitter). I recommend that you read his four part series on the topic starting here.

With all of the COSPLAY going about here in the States, I find some of the scarier things to be about either stuff I have previously fixed or about which I have written or tweeted or all of the above.

Take for instance this article about the blatant disregard by some vendors and clients in regards to security. I still can’t figure out why the public role would ever need to be dbo for a database – at least not a legitimate reason.

Or we can take on the recent time I tweeted about a cursor that I fixed. I took that scary cursor down from a 30+ hour run time to a mere 50 seconds. Here is a segment of the execution plan (plan is roughly 4mb in size to give a little scale) zoomed out to 1/5th.

The query was much uglier than that originally. Imagine that beast looping through on your server for 30 hrs, and that is not even the entire thing. It is little wonder why things started to drag on the server.

Another scary item I like is the effect of implicit conversions. That is a topic that can be viewed pretty easily through the use of google-fu. Here is a short demo on the implications of implicit conversions.

[codesyntax lang=”tsql”]

Transact-SQL

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

CREATETABLE#T1 (SomeIntINT,SomeRealREAL)

CREATECLUSTEREDINDEXT1_SomeRealON#T1(SomeReal)

CREATETABLE#T2 (SomeIntINT,SomeRealREAL)

CREATECLUSTEREDINDEXT2_SomeIntON#T2(SomeInt)

CREATETABLE#T3 (SomeIntInt,SomeRealNVarchar(Max))

--name mismatch and datatype are intentional

CREATECLUSTEREDINDEXT3_SomeIntON#T3(SomeInt)

GO

SETNOCOUNTON

DECLARE@IINT

SET@I=0

WHILE@I&lt;10000

BEGIN

INSERT#T1VALUES(@I,@I)

INSERT#T2VALUES(@I,@I)

INSERT#T3VALUES(@I,@I)

SET@I=@I+1

END

SETSTATISTICSTIMEON

/* Query 1

No implicit Conversion */

SELECTCOUNT(*)

FROM#T1T1INNERLOOPJOIN#T2T2ONT1.SomeInt=T2.SomeInt

OPTION(MAXDOP1);

GO

/* Query 2

Pretty bad implicit Conversion */

SELECTCOUNT(*)

FROM#T1T1INNERLOOPJOIN#T3T3ONT1.SomeInt=T3.SomeReal

OPTION(MAXDOP1);

GO

SETSTATISTICSTIMEOFF

[/codesyntax]

In this demo I have created three temp tables. Each is pretty simple in nature and each is to receive 10,000 records. The insert statement just inserts an integer into each field of each table through the while loop. Notice that I intentionally named a column in #T3 to be SomeReal but the datatype is an NVARCHAR. This is to underscore a pet peeve of mine that I have seen over and over again – naming the field in the table after the datatype and the datatype doesn’t even match.

When this query runs, I get the following timing results.

The thing that stands out to me is the huge difference in time between the implicit-free query and the query replete with an implicit conversion. The implicit conversion query

was about 930 times slower than the query free of implicit conversions. Granted that query was against a cold cache, so let’s see what happens to an average of five runs each against a warm cache.

With a warm cache I see an average of 51ms for the implicit free query. On the other hand, the implicit conversion query runs at an average of 84525ms. That equates to about 1644 times slower.

Sure this was a contrived example. But keep in mind the table sizes, the datatypes and the number of records in each table. Had this been a more true to life example with larger tables and millions of records, we could be seeing a query that is far more devastating due to the implicit conversions. Let’s just call it the grim reaper of your database. (Grim Reaper from www.mysticalpassage.com/grim_reaper.html)

With these horrifying things to haunt your database, I leave you with this new hope as you battle the dark side and grim that is in your database.

Today, we have a special Halloween edition. For me, Halloween and computer geek go quite well together. And thinking about it, I wanted to try to better understand if there was a correlation. As a DBA, have you wondered the same thing?

Well, I have a short list of five things that may help you to correlate your affinity for Halloween with your love for Databases.

Tombstones

Did you know that a tombstone is a legitimate thing in SQL Server?

Tombstones are replica related. They are deleted items in the replica and are used to make sure the deleted item doesn’t get put back in the replica inadvertently.

Tombstones are not unique to SQL Server. These are commonplace in Active Directory as well.

Ghosts

Not all rows that are deleted move on to the afterlife quickly like they should. Some like to hang around due to unfinished business.

The unfinished business in this case is the server running a cleanup thread. This has to be done when the server is not too busy and has enough free resources to help these records move on to the afterlife.

You can see the evidence of these ghosts with specialized equipment. By the use of a DMO, we can see the ghost record count on a per index basis. The DMO is sys.dm_db_index_physical_stats. Take a look at the ghost_record_count column in the returned record set.

With more specialized equipment, the engine takes care of the cleanup and removal of these ghosts. Here is an in-depth foray into the world of SQL ghost hunting. Whatever you do, don’t cross the streams.

Zombies

It’s alive!!

No, I killed it!!

It can’t be…How is it still alive?

The transaction will not commit and may get rolled back. The zombie has reared its’ ugly head. A transaction that cannot commit but keeps going (or rolls back) due to an unrecoverable error is a zombie transaction.

Children of the Corn

Well, this one is not really something in SQL server. That said, every time I think of orphaned users in SQL server – children of the corn comes to mind.

An orphaned user is one in which the login SID does not match for one reason or another. This makes it so that the user can no longer log in to SQL server.

If you don’t know about these kids, they can really make for a frustrating day. Read more here.

Bonus

I have just covered five things in SQL server that correlate quite closely to Halloween. But this by no means is an exhaustive list. For instance, an obvious correlation is the “KILL” command. Another good one is the monster known as the Blob (read more about that monster here and here).

With the opportunity to have Halloween every day, it’s no wonder I like being a DBA.

Auto-generated statistics names can seem like they are entirely random, but there is a method to the madness. With a little effort and a bit of TSQL trickery, we can decode those names and reveal what the names really mean.

The default collation for SQL Server is a pretty bad idea. Sure, it works but so does SQL Server 7. When you have the opportunity to update to more current technologies, it is a good idea. Sometimes though, that upgrade can come with some pain. This article shows how to alleviate one such pain point by fixing problems related to collation conflicts and XE.

This article has just shared multiple tools to help you become more acquainted with the Query Store! This acquaintance is coming via an extremely powerful tool called Extended Events. Through the use of these two sessions and two additional scripts, this article demonstrates how to become more familiar with the internals for QDS.

The ability to quickly and easily trace a query is important to database professionals. This script provides one useful alternative to trace a specific spid similar to the method of using the context menu to create the trace within SSMS and Profiler.

This article takes us to the edge with a couple of CRM related errors after changing the service account to a more secure Managed Service Account. Despite the CRM reports working properly within Report Manager (via SSRS), the reports would fail in CRM.

Working with Extended Events will help you become a better DBA. Working with PoSh can also help you in many various tasks to become a better DBA. Combine the two and you just might have a super weapon.