Introduction

In this article I'll address some discoveries I've done in Microsoft's implementation of the CLR and how to take advantage of them, and what risks they involve as far as I know.

Background

I'll keep two parts of the topics I'll discuss about, the first will be the explanation of the problem, the
second is the code implementation in MSIL or C#. I will point here too that I don't mean you to use this because of its risks.

So I expect some knowledge about what is MSIL and how to compile it with ilasm.exe atleast.

I also would like if anybody among the readers can test this stuff in Mono and share his experience.

Problem and solution

I've walked to some limitations about the use of pointers in C#, most of then I though where crazy thing, until I
needed them. The problem was that I had wrote a large lib full of unsafe blocks and it was becoming a problem
to maintain.

The way I solved it was based a discovery I did thanks to CodeProject, and once I had it up and running, I wanted to
be able to do all those crazy thing I had in my mind... and guess what? they can be done! I'm not using all that
stuff soon, and since it was thanks to CodeProject, I'm giving it back to the community.

To give a raw idea of what we are dealing with, I'll say that you will be able to do generic pointers and also
some arithmetic forgiving about the type and one or two details more you may have forgot about.

Points of Interest

As I said before we will handle some weird things about pointers and types, in general we will explode as deep as
we can unsafe code... about that, there is people that miss unsafe with unmanaged, and that's not true. The unmanaged code runs outside of the CLR such as P/Invoke. Unsafe in the other hand runs inside the CLR this means that you don't have a perfomance hit of leaving and returning to unmanaged code, but also that It's not that fast as it's unmanaged equivalent.

I have to confess that I thought about using protection glasses in case the the CPU explotes or something, but
since I'm writing this you can be sure that the risk is not that big. But surely we will do things that may end in
the app crashed like in the days of VarPtr in VB 6.0.

If we go back to that age... we had VarPtr, ObjPtr, StrPtr and a weird hack named ArrayPtr, Let's start doing them
in C#, I know there are solutions for mashalling, but I don't mean that. I want to mention only for historical reason that there was a ProcPtr too. And to note that we will not go into ObjPtr directly here, but I can tell that VarPtr will also do what ObjPtr used to do.

Why the GC can't collect strings?

First off, the GC can collect the objects, I mean the instance of the System.String class that points to the string's
content. It is able to move it around the heap and collect it like any other object. Period.

What the GC can't do is to collect the memory of the string it self when this is interned, I mean the palce where
all the characters are stored for an interned string. Does that mean that it can't collect any string? no. It only
means that it can't collect interned strings, and those includes the string literals of .NET. Also string literals may be
frozen strings more on that later.

Now what's a interned string? for those who had the chance to work with Win32, you probable know about something
called ATOMs, the ATOMs was objects (in the abstract of the age) that allows us to store and handle strings. The
ATOMs was good idea to store commonly used strings, such as the texts of the windows of your application. The good
news for the developers was that they only needed to store the string once. All the strings with the same text
actually pointed to the same area in memory, making it less expensive in memory space. Also this string memory was
managed by the operating system, and any proccess could take advantage of the text there.

Ok, that's exactly what System.Intern does, I can't tell if that uses ATOMs, but that it behave the same way, and
thus all the strings with the same texts points to the same location in memory (may be memory managed by the OS (...) we may ask people from Mono to see how they did it). Think about what it takes to recover
that memory with GC (...) you would look for all the string that may point to there, and that includes strings that
points to partial parts, and that overlap (...) so GC just doesn't care about internal strings. But surely it can
recover the rest of strings, such as those created with StringBuilder, that's why using StringBuilder for string
manipulation is a good practice encoraged by Micrososft. Remember that you can call Intern on any string.

Talking about StringBuilder, there are reasons for the strings to be immutable, the first reason comes from the need to get sub-strings. If you need to get a sub-string and strings aren't immutable, what would happen if you point to a internal part of another string and that one moves to another place or changes it's contents? The answer that the your sub-string will be damaged and you don't want that.

So languages like the old VB created a copy of the string to give the sub-string, this behavior makes easy to do concatenations, replace and write. What's the problem with that? It's that both sub-string and concatenations operations requires to copy memory, so the designers decided to make then immutable so the sub-string case will be faster. But that leads to a problem when you write (Called C.O.W, copy on write) and concatenation will be less efficient.

Today we have StringBuilder to solve that part. between others adventages of immutable strings we can find that they are supposed to be thread safe and that they can be interned saving memory.

With that said, consider what it would involve to use StrPtr with a non internal string.

1) How to do StrPtr

I have not found any undocumented risks on this code. The only failure case I know is passing null as argument,
in which case you will get 0 as return value. If you try to read or write a pointer pointing to 0 you will get
NullReferenceException.

First, yes it's dangerous, secound, no the GC will not take the string.

Another thing... if what you need is a byte array with info of a string you can delegate that work to Encoding
types in System.Text namespace, if you need a char array, use String.ToCharArray().

I have had complains about StrPtr, so I'll talk about those risks I
thought you already knew:

In the above code main calls the funtion
Test
which declares and initializes a string named
a
and then retrieves the pointer with the StrPtr function showed above, the next step is to call the garbage collector to get that string that just went out off scope garbage collected (something that actually will not happen). The next step is to read the area where the pointer was looking for, and the output
is the following:

S
O
M
E

Which means that the string is till available. Ok, that looks ok, the next step was to repeat the experiment with a
larger string (it was a copy of the code above). and the output was:

Do you notice that call and those local variables? the only thing I will say about it is that, if you though
you skipped using .NET framework functions using fixed to get the data of a string instead of ToCharArray(), you
are wrong. Also note that this implementation may change in the future.

What do you think the output will be? if you said "SOME", you are worng, because strings are inmutable,
.NET takes the freedom to share the same pointer among all the string literals that are equal. So the output
is "NULL", to avoid this use StringBuilder class from System.Text namespace which will prevent your
string to be interned.

2) How to do VarPtr (and walkthrough)

This is actually a documented one. You can use use
&
to get your pointer, just like in C++ (in syntax atleast), just remember to be in a unsafe block and had /unsafe option on. But can
you write
VarPtr function?

Before going to write the funtion, let me repeat that we are doing something dangerous, I don't want to be blamed because you forgive that.

In order to create this function keep in mind that you need to use a parameter by-reference, because if you use
a by-value, the whole idea is nonsense. Once that said, the problem is simple: if you write a funtion that takes
a value to retrieve it's address, what type will you declare the parameter?

- If you said object, you may think you will have to deal with boxing - unboxing, but actually the compiler will
just say: "can't conver to ref object".

- If you said the same as the value, then you will have to write the function for each posible data type. That isn't the idea.

- If you said to use a generic type then you have got the idea, now you how to create a generic pointer?

For long I though it was unpossible, the reason is that the following code ends with two CS0208 errors,
"Cannot take the address of, get the size of, or declare a pointer to a managed type ('type')":

It's included in a library I called "GenericPointers" and in a class named "PointerBuilder". The
source code in MSIL is attached to this article.

Now using this Lib compiled from MSIL, I'll need to change:

int c = Ptr(a);

with:

int c = (int)PointerBuilder.AddressOf(ref a)

The result was the following:

70642928706429282

So this is out VarPtr, but in order do create it we discovered that Microsoft's implementation of the CLR supports
generic pointers... what else could we do with that?

About the risk. you will have the same that you have with any pointer, so spectacular crashes are up to you.

3) How to do ArrayPtr?

There was a problem in VB 6.0 and it was that the arrays didn't allocate their elementes where the pointer you get
with VarPtr points... so people managed to get the address to that point using a "hack" they called
ArrayPtr. In .NET happens the same thing with arrays. I coded a solution in MSIL, which I'm not including here,
because after testing it I discovered that the procedure Ptr I showed in numeral 2 gives the address for Strings
and Arrays of any type.

These are news, array didn't pass all the tests, so if you are accessing an array make sure to have a copy of the array too, because the GC may take the array.

4) How to do a "StuctPtr"?

There has been a lot of people that has asked how to get a pointer to a struct they declare in C# and handle
it byte by byte, I solved this problem too, the implementation in MSIL is the following:

This routine will allow us to convert a pointer to a generic pointer, with this power we will convert a pointer to
a byte array into a pointer to the struct we want, and then asign to it's value our struct. a sample code to
take advantage of this in C# is the following:

Ok, now beware of passing a byte array wich length is less than the size of the struct, if you need a sizeof for
generic types there is one in the MSIL code I'm including with the articule.

The reason because of which you may need a generic sizeof is that if you use
sizeof in a generic type it will say error CS0233 "'identifier' does not have
a predefined size, therefore sizeof can only be used in an unsafe context
(consider using System.Runtime.InteropServices.Marshal.SizeOf)". I've told this
to Microsoft and there is risk that sizeof will work on generics with struct
constrain in the future.

About taking out the struct constrain is up to you doing this with reference types, do it at your own risk. I'm not
interested myself in such situations, since anything you may accomplish with that may not work in another CLR
implementation form Microsoft or third party.

I want to note that I've done tests with garbage collection and I haven't found any problems, looks like GC is
smart enough to know that there are more than one thing allocated in the given memory space, just as happens in the
C# version of union. I'll post here in case you forgot about it:

The code above shows how to use unsafe with unions to get the bytes of a
given sturct. This code will not work in generic version, because it will
need to have a variable size of the field data, and it will
response with an error CS0133 "The expression being assigned to 'variable'
must be constant". (and also you will need that sizeof).

5) How to do Array to Array copy?

We all now that we can use CopyTo to copy info into another array of the same type, but what do we do if we need
to copy the binary data to an array of another type? (I don't mean to convert the items). So far we have seen how
to handle some weird pointers, now I'll show an example of a funtion that makes memory copy much faster (no need
to walk with a pointer all the array). The code is the following:

The star in this code is Marshal.Copy which will do the work we did with RtlMoveMemory (CopyMemory) in those
"hack" days of VB 6.0 if you are looking forward to graphic work, Marshal.Copy and LockBits are a
good team.

This code is wrote to work as extension methods, if you are uilding to .NET 2.0 add the following code in order to
make it run without changes (again in case you forgot about it):

6) How to do generic arithmetic?

If you have done a vector type, a list type or a matrix type, you may have came to the problem of generic
arithmetic. First off, What's generic arithmetic? Generic arithmetic is a term to refer the feature you need to
add, multiply, divide, substract, negate, shift ot get the remainder of a division or any other arithmetic
operation derived from those with numeric values without caring if they are int, short, long or whatever is a
numeric type.

Now that granted, when do you need it? It's needed in such rare but useful situations that we have learned to
think they are imposible. For example consider writing a function that calculate the average of the numbers in
a generic enumerator, do you want to write it once for each numeric type? of course not, you want to have a
single generic function handle the problem, this is the scenario for which I have the solution (except for decimal
an any other numeric not primitive type).

The idea is quite simple, now that we have seen how to use MSIL to handle generics, what happens if I tell him to
add two values of a generic type? for instance:

The code above uses add to sum up the values a and b,note that add does not check overflow, to get a overflow
checked version change add with add.ovf. the others operations are done unther the same principia.

Guess what? it works! but it's pretty, pretty dangerous, if you happen to pass objects, strings, or even decimals,
it will end with an AccessViolationException or with a more expectacular FatalExecutionEngineError, which is
a fatal error, in other word it crashes your code not matter how many try-catch statements do you have.

Ok, to avoid this danger include the following test around this code:

if (typeof(decimal).IsPrimitive)
{
/*...*/
}

I would had include it in the MSIL, but checking this again and again may run in a bottleneck.

Again for curious readers, here is the same done from C# (it ends with a CS0019 error "Operator 'operator'
cannot be applied to operands of type 'type' and 'type'):

publicstatic T Ptr<T>(T a, T b)
{
return a + b;
}

To close the matter of generic arithmetic, I want to add that there is another
way to acomplish gemeric arithmetics. It's slower but more secure and will
require LINQ, the following code is a working example of this approach:

Disclaimer

There must be a question around, and it is, why do I say that Microsoft didn't want us to know?, well It happens
that this things has been asked Microsoft to be added to .NET or Visual Studio, and MIcrosoft responded
with a "won't fix - by design", may be even they don't know they did this, but if they know, they didn't
want us to know. Let's try to think why? It wont be actually unsafe to do generic arithmetic if they had support for
a "numeric" generic constrain about what I want to quote this:

"

Bruce Eckel
: Will I be able to do a template function, in other words, a function where the argument is the unknown type? You
are adding stronger type checking to the containers, but can I also get a kind of weaker typing as I can get with
C++ templates? For example, will I be able to write a function that takes parameters
A a and B b
, and then in the code say,
a + b?
Can I say that I don't care what
A
and
B
so long as there's an
operator+
for them, because that's kind of a weak typing.

Anders Hejlsberg
: What you are really asking is, what can you say in terms of constraints? Constraints, like any other feature,
can become arbitrarily complex if taken to their ultimate extreme. When you think about it, constraints are a
pattern matching mechanism. You want to be able to say, "This type parameter must have a constructor that
takes two arguments, implement
operator+
, have this static method, has these two instance methods, etc." The question is, how complicated do you want
this pattern matching mechanism to be?

There's a whole continuum from nothing to grand pattern matching. We think it's too little to say nothing, and the
grand pattern matching becomes very complicated, so we're in- between. We allow you to specify a constraint that
can be one class, zero or more interfaces, and something called a constructor constraint. You can say, for example,
"This type must implement
IFoo
and
IBar
" or "This type must inherit from base class
X
." Once you do that, we type check everywhere, at both compile and run time, that the constraint is true. Any
methods implied by that constraint are directly available on values of that type parameter type.

Now, in C#, operators are static members. So, an operator can never be a member of an interface, and therefore an
interface constraint could never endow you with an
operator+
. The only way you can endow yourself with an
operator+
is by having a class constraint that says you must inherit from, say,
Number
, and
Number
has an
operator+
of two
Numbers
. But you could not in the abstract say, "Must have an
operator+
," and then we polymorphically resolve what that means.

"

Sorry by the long quote, but the context is importat here, for abstract it implies that we don't have support
of numeric constrains because of the complexity involved in such expesific constrain. now think about the code I
show here... it does it! but it have strong risks, which I'm afraid I may have not found them all. I not saying:
"Go hack .NET", I'm saying to Microsoft to support this in safe way, and to the developers to use this
carefully, I want to show to Microsoft that we may need this featues in the language and CLR. Another reason why
I'm not saying to hack .NET is that I like the idea of having code running in tird party implementations, and if
we want that then hacks are nonsense.

After that another question may have raised, and it is what is all this useful to? I can't go the specific kind
of work we may need it (and remember that I said that this wasn't for marshalling), First use I may do with this
is all that work we do reading stuff from streams, secound would be working on graphics or sound for realtime
applications, and derivated from that one another use is video games, ok you will tell me two things on the matter:

1) it's not secure

Like any other way of doing things there are good practices and mistakes to avoid, I suggest to work with
this only inside a component, and test, test, test.

2) there are other ways to do it

I know, "don't reinvent the wheel", but remember that sometimes it's good to know how the wheel
was done, and that wheel was done by humans so it's not perfect and there is room to improve. So I have two things
to say, first is that I will not go to C++ to have a more unsecure scenario just because I want to handle
some dangerous things, nothing more away from my scope, I want to keep the security up in any place I can
while getting enough speed and power to accomplish my requirements. And secound is that even as if I know that
there are third parties products that solve my problems, some developer worked with that, and did the dirty work
for you to have the hands clean. What if I tell you that I'm heading to be one of those? or if you are
needing some dirty work for first time, and there is not third party product? so atleast think about it.

Lastly if you are still thinking that this doesn't help at all, remember that you know that it's posible thanks to
me, so you may take this article as a curiosity expo, and skip sending me messages saying: "what is
this for?".

Legal note

This article is published under the Creative Commons Attribution-Share Alike license, since this license
says: "Any of the above conditions can be waived if you get permission from the copyright holder.",
I'll allow you to forgive the "Share Aike" part, releasing this license from it's viral nature. I only
care about those that doesn't create derivate product that use this code for commercial use to include my name in
the "special thanks"

[I posted this earlier but then some idiot approved the article so now I'm posting it again.]

What do you mean by "What Microsoft didn't want you to know"?

I suspect that you need to rework whatever you're working on and either learn .net better or stick with C/C++.

For instance, I see no reason to have a method that returns a pointer to a string. If you need to do something strange with a string, then just do it within one unsafe block; don't pass the pointer out of the block. You should also have pointed out that the length of the string is stored in an int just before the first character. See this[^] and this[^] for some things I've written to access strings in unsafe blocks. Neither is intended for actual use, there are better ways, and neither passes the pointer to the string out to be processed by something else, they just do it.

1) If it's about to CharArray see what it does in MSIL, and compare it to what the pointer got with fixed does. Ok, CharArray is more secure. but not say that I didn't know, more even I give it for granted you all know it.

2) about passing the string to a byte array, I mentioned to use Enconding from System.Text... dam, why did I set this article to "advanced" at all?

3) Ok, about your F funtion, I agree that passing pointers away from a proceduere is not good thing (still your code can be enhaced with Marshal.Copy)

4) I'm here to say, it's posible, I don't expect all the people going to use this StrPtr dancing in a flower garden. Do you want to talk about stackalloc and how to read the stack taking adventage of that pointer (I don't dare test write)? complain to microsoft to leave this posible. .NET is not a paradise even if it looks like it. and about stick with C++, that's my problem not yours.

5) If some idiot approved the article, some other idiot got your earlier mesage in his inbox and added a disclaimer, and some third other idiot didn't read it. Look for it to see why I said that Microsoft didn't want you to know. The proof of that we are talking about idiots is in the fact that you don't like this article and wanted to prevent it's publishment and still can't understand why Microsoft didn't want it to be public.

I don't know, I don't think there's any concensus about what the levels mean, I just mark everything "beginner".

Theraot wrote:

why Microsoft didn't want it to be public.

You still haven't explained that. I don't think there's anything in .net that Microsoft doesn't want public (other than the source code).
And of course through Reflection even private members are public, so one could argue that everything in .net is public.

You could possibly change it to say, "Here are some dangerous and unnecessary techniques that are not recommended practice".

I added some tests on the StrPtr I hope helps to understands it's outgoing. I also hope I can tell this articule is more mature now.

From now on, I'm taking all the complains as "your article is interesting enough to complain about it" and I'll to answer.

We are not going to start a war, right? so tell me what else you don't like, and I'll see if I can add some material that the readers can take as warning or reference. But I'm not taking out what is already there.

I am not exactly sure what the point of this article is, but the techniques described here are not recommended for any practical use. This article depicts some dangerous use of unsafe code in C# that generally bypasses the protections that should be used when doing acceptable unsafe coding. Pointers will no longer be pinned once their fixed block exits, which could pose some serious problems to whomever uses the pointers returned. The capabilities described, such as converting a pointer to an int, already exist within .NET via types such as IntPtr (which has two functions to convert pointers to Int32 and Int64) and Marshal (which has functions to retrieve pointers to and convert between a wide variety of data types, managed and unmanaged).

If you need to directly manipulate memory in .NET, you should be using the existing facilities, unsafe methods, fixed blocks, and a dose of common sense in regards to how .NET works (its a garbage-collected runtime that WILL move items around in memory as it sees fit without hte use of a fixed block). Don't use the techniques described in this article. C# is not C++...pointers are available when nothing else will work, but general use of them is not a proper way to use this language.

In general, don't use pointers whenever possible, and avoid them at all costs, so that you can gain the full benefit of what a garbage-collected runtime has to offer.

Your updates have not changed anything. Garbage Collection is non-deterministic...you can not know when it will actually collect something, even with a forced collection. Your examples are also relatively simple, with practically ZERO load on the GC. You create a single pointer, then try to force the GC to collect (which may or may not actually cause a GC to run immediately...it just requests it). However, collection of a non-volatile heap such as yours is unlikely to change memory in any significant way. Even if a collection happens, you are not allocating enough objects to be sure thay the chunk your pointer points to will be overwritten...so creating a new pointer of the same size without nulling the original is likely to change data in the same location in memory, and appear as if it is working.

The entire point is that the GC is non-deterministic. Even using the GC static class in the runtime does little to offer any kind of guarantee with the GC or the state of memory when you have such a low load on the GC. I'm sorry, but there is very little you can do to convince anyone who understands the point of a garbage-collected execution environment that screwing around with pointers like this is something that should be called "hard core microsoft .net".

Its even unlikely that most of us will think that an article that describes such techniques is useful in any capacity other than opening a dangerous door to all the wannabe programmers out there who will latch on to this concept and try to use it with disasterous consequences in real-world scenarios. Anyone with a shred of talent with .NET will recognize the dangerous potential of these techniques and avoid using them.

ok (...) but, I still have something to say about GC, and it's that I don't just request garbage collection when I call it with "Forced" as argument. MSDN says about it:"Forces the garbage collection to occur immediately."
Check it here:http://msdn.microsoft.com/en-us/library/bb495757.aspx[^]
At the time being, I've done another test, and it is to run the following code for 1 hour (and counting):

How do you explain i can have running for that long even with the use of Forced? You can take 2 position on this from my point fo view, the first is to say that this may not work in incomming versions, and that this is actually a bug that Microsoft should solve soon. And the secound is that the documentation is wrong and Forced doesn't do that. In both cases I would like to ask you to submit this feedback to Micrsoft in Connect.Microsoft.com[^] because that's what I've came looking for, I mean to push Microsoft. And I want to leave clear that I don't suggest this to be used on production and that I agree that this is dangerous.

Try that code and check the documentation. It got to be that documentation is worng (Forced doesn't work, or String are safe from GC) or that GC is worng. Please go tell Microsoft. I would wish I would do everybody go tell Microsoft. About the StrPtr Microsoft should add a warning when converting a pointer to another thing that's not a pointer or viceversa, and another when putting a pointer as return type, go suggest that too in Connect. I vote for break the code of those who uses this, and give an also alternative in the CLR and/or the language for generic arithmetic and what I show in StrucPtr. And do it as soon as posible.

How do you explain i can have running for that long even with the use of Forced?

You still don't quite seem to understand. You are not putting any load on the GC. Even if a collection does indeed happen (i.e. you force it...another bad practice, I'll get back to it in a moment), you are not putting anything else on the heap. This means that the data will probably remain where it was...there is no pressure to move it somewhere else. Even in the event that you create other objects after creating your string, as long as there is a reference to your string, it won't get collected (and as long as there is no reason to move that chunk of memory somewhere else, the GC won't). I totally expect, given the way you have written your little test, for the pointer to keep pointing to a location on the heap that will contain the data you put there...there is nothing in your test that would cause it to be moved or overwritten.

Theraot wrote:

It got to be that documentation is worng (Forced doesn't work, or String are safe from GC) or that GC is worng.

Your missing the fourth option...that your test is insufficient to force a specific outcome from a non-deterministic process.

Theraot wrote:

Please go tell Microsoft.

No...there is nothing wrong. The CLR, C#, and unsafe code work exactly as they should.

Theraot wrote:

About the StrPtr Microsoft should add a warning when converting a pointer to another thing that's not a pointer or viceversa, and another when putting a pointer as return type, go suggest that too in Connect.

Microsoft has been very clear about the use of pointers in C#. They have a keyword called unsafe for christ sake...if that doesn't tip you off, I don't know what will. As for suggesting anything to Microsoft...there is nothing wrong, and therefor nothing to report. This stuff works as designed.

Theraot wrote:

If you don't agree with me that this is Microsoft fault, then let everybody say it was mine, I have Creative Commons Licence, so you all can tell this is my fault.

To settle this issue once and for all, because Microsoft is not at fault here in any way. They have been very open and clear about the use of pointers in C#. Please read the following excerpt taken strait out of the C# Programming Guide on MSDN:

"Be aware that passing pointers between methods can cause undefined behavior. Examples are returning a pointer to a local variable via an Out or Ref parameter or as the function result. If the pointer was set in a fixed block, the variable to which it points may no longer be fixed."

Emphasis has been added by me to highlight the critical pieces of information here. First note..."undefined behavior". That means that returning a pointer from a function that has a fixed block has no expected (or deterministic) result. It may work. It may not work. It may work for a short while, then stop working. It may work most of the time, and not other times. It may even work one way in .NET 2.0, and work a completely different way in .NET 4.0. The point is, the behavior is UNDEFINED. Second thing to note...the documentation explicitly states that returning a pointer via any of the three possible methods, be it out or ref params or return, can have unexpected results. It explicitly states that the pointer may no longer be fixed. It doesn't state when it will stop being fixed, but whenever the pointer is no longer fixed, there is the POTENTIAL for the data it points to to be moved by the GC. Again, were talking about a non-deterministic process...so there is no way to predict or even force (given the unpredictible nature of when the CLR unfixes a pointer) the particular scenario of a pointer suddenly pointing to invalid data.

Plain and simple, your article describes something that is clearly and definitively defined and described as being UNDEFINED and therefor unpredictable. Microsoft explicitly states in their documentation that you should be aware of the consequences of returning a pointer from a function. Not only that, there are several other caveats about using pointers in C# that Microsoft has also been very clear about.

You talk in your article and in your replies as if this is something unknown, "hidden" by Microsoft, and therefor worthy of exposure, which will, I assume you hope will lead (given the nature of your article and your replies), to the subsequent prosecution of Microsoft for their haenous C# pointer crimes.

I'm sorry to burst your bubble...but nothing you have described is unknown. Nor is any of it unexplainable. Even your test, which seems to portray that returning a pointer is perfectly OK while according to Microsoft it is not...is explainable. It is possible to describe exactly why your encountering the behavior you are encountering with your example so long as you properly understand how the .NET GC works (as I have done above).

Fact of the matter is...pointers in C# are not the same as pointers in C/C++. In C++, when you allocate memory, that memory is fixed by default. Any time you allocate memory in C++, you are responsible for freeing and cleaning that memory up. You and only you. There is no background process that could come along at any point in time and move your chunks of allocated memory around or clean them up for you. In C#, however, there is a background process that is the primary force behind allocating and freeing chunks of memory. In addition, this background process DOES indeed move your allocations around for its own purposes. You can't treat a C# pointer like a C++ pointer. The proper treatment of a C# pointer is well described, and clearly stated as generally "unsafe"...used at your own risk if you so choose. It is STRONGLY recommended that you do not use pointers in C#, and use the language how it was meant to be used...as one that has automatic memory management.

Runned until I got a OutOfMemoryException. I know you can still argue about the GC. don't worry I notice that discuss this with you will not take me anywhere.

About the cite, didn't you talk about wannabes? a warning is better, I will still suggest that warning to Microsoft.

Jon Rista wrote:

I'm sorry to burst your bubble...but nothing you have described is unknown. Nor is any of it unexplainable. Even your test, which seems to portray that returning a pointer is perfectly OK while according to Microsoft it is not...is explainable. It is possible to describe exactly why your encountering the behavior you are encountering with your example so long as you properly understand how the .NET GC works (as I have done above).

I agree almost all that, I didn't say that it was unknown neither unexplanable. The earth isn't flat and the GC is not getting the strings (at least not the intern ones) and I failed to convince (I know you will say that that's because it's false, well, I disagree). And yes, I hope Microsoft changes the behavior of this and I'll be gald to post it here when that happens.

I'm using a WeakReference, It was about that the GC and the string. I say that the GC is not taking the string, neither moving it. and that's why in the case of the pointer it is not becoming invalid. something that should be, I accept that. Thinking about it I posted it too lightly at the beggining. It's not suggest to use it, it's that I show that that work (ok, it shouldn't).

I think I can do an abstract to all this and it's that we are trying to explain the same thing in two diffent ways that neither of us can 100% proof it. You say that the GC will take the string, I just haven't reached the situation so far. I say that it will not.

Like saying that if you go the horizon in the sea you wil reach the world's waterfall, and if we haven't reached it yet it's because we have to travel more. Or from the other scope that if we go far west we will reach india, but everybody says we will find the world's waterfall.

I don't want to go disassemble CLR's internal calls (I don't have the tools neither currently). So at the end it's your word (and that one from a lot of developers) against mine.

It describes illegal usage of unsafe pointers without proper pinning of objects during pointer scope.

The clr only ensures that the object remains at the same native address in memory as long as you are inside a fixed block or are holding a GC-Handle to it.

I (and microsoft too - just read the documentation about the fixed-statement and GCHandle) therefore highly disencourage the practises discussed here! It will sooner or later result in strange access violation exceptions or in overwriting of essential data in other objects which both are very hard to track down.

You are always ever allowed to use an unsafe pointer inside of the fixed block or if you are using a GC-Pinned-handle, as long, as the GC-Handle remains referenced, because after the gc-handle goes out of scope it may also be garbage-collected and the referenced object may also again be moved around in memory by the Garbage Collector anytime, and your pointer will no longer reference your object, but point to some other memory region.

Sorry to be such harsh, but this is really dangerous.

(sorry if this is now a dupe, but the first vote did not show up and I wanted to make sure nobody tries to use the methods described above)

"Don't use in production", it's ok to do some experiments and do academic research some time, by the way, I just added a disclaimer I hope have the time read.

And thanks about the note GC-Handle, I missed to point that.

Let me tell you something else by the way... a message will not prevent people from using this code, I hope there is people smart enough to understand the risks and even to handle them. for instance you can use critical sections to pervent garbage collection in the time you need the pointers alive (just as example).

Remeber torvalds: "If you think your users are idiots, only idiots will use it." and god I don't want idiots using this.

thank you for not being angry because of my rant - because I did not want to offend you, but I'm getting near-heart-attacks when thinking about thousands of people applying such practises in their programs

My first try on this comment was much longer (and a bit more polite, I have to admit ), but it got lost on it's way to the server, so I had to do it a second time, which did not improve on my patience

I think adding such a disclaimer would really make things better. Becauase except for the described methods I think it is a interesting and well written article.

But after initially reading your article I did not have the impression that it was just academic theoretical. In opposite, I had the impression (especially because of VB6-like methods like VarPtr) that you wanted to show people ways to circument the managed memory management with it's not immediate but really disastrous consequences when used in production code. It really has a reason, why microsoft does not tell people to do such things