FILE * fopen (const char *__restrict _name, const char *__restrict _type);It would be nice if it was possible to somehow get the parser to expand that particular macro, even if it's just a regex somewhere. I don't really expect it to be able to follow the newlib headers & pick the right macro to expand.

- if (token == _T(","))+ // comma is a delimit only it is not wrapper by ()+ if (token == _T(",") && level == 1) { results.Add(piece); piece.Clear();

You need to add a replacement rule in CC setting: _EXFUN -> @

And the result can be shown in the screen shot below:

Logged

If some piece of memory should be reused, turn them to variables (or const variables).If some piece of operations should be reused, turn them to functions.If they happened together, then turn them to classes.

If some piece of memory should be reused, turn them to variables (or const variables).If some piece of operations should be reused, turn them to functions.If they happened together, then turn them to classes.

If some piece of memory should be reused, turn them to variables (or const variables).If some piece of operations should be reused, turn them to functions.If they happened together, then turn them to classes.

Today I was about to submit my patch for function calltips, and coincidentally found this post. My patch adds calltips support for some complex macro functions, and variables of function pointer typedefs. Both are required to support the glew functions.

If some piece of memory should be reused, turn them to variables (or const variables).If some piece of operations should be reused, turn them to functions.If they happened together, then turn them to classes.

But I don't have tip info if the mouse hover on the "foo(5)".Find declaration of "foo(5)" gives nothing.

Logged

If some piece of memory should be reused, turn them to variables (or const variables).If some piece of operations should be reused, turn them to functions.If they happened together, then turn them to classes.

// NOTE: ReplaceFunctionLikeMacro will recursively expand all defines and macro calls, // but will not expand macro names, which is what we want smallTokenizer.ReplaceFunctionLikeMacro(tk); tk = tree->at(tree->TokenExists(smallTokenizer.GetToken(), -1, tkFunction|tkMacroDef|tkVariable)); if (tk && smallTokenizer.PeekToken().empty()) // only if the expanded result is a single token token = tk;What does the "will not expand macro names" in comments means? as I see, ReplaceFunctionLikeMacro() did a full expansion until nothing is expanded.

« Last Edit: September 07, 2014, 04:45:23 pm by ollydbg »

Logged

If some piece of memory should be reused, turn them to variables (or const variables).If some piece of operations should be reused, turn them to functions.If they happened together, then turn them to classes.

// NOTE: ReplaceFunctionLikeMacro will recursively expand all defines and macro calls, // but will not expand macro names, which is what we want smallTokenizer.ReplaceFunctionLikeMacro(tk); tk = tree->at(tree->TokenExists(smallTokenizer.GetToken(), -1, tkFunction|tkMacroDef|tkVariable)); if (tk && smallTokenizer.PeekToken().empty()) // only if the expanded result is a single token token = tk;What does the "will not expand macro names" in comments means? as I see, ReplaceFunctionLikeMacro() did a full expansion until nothing is expanded.

#define DEFINE MACRONote that there is no "()" after MACRO.. so this is not a valid "macro call", just the macro name is used alone. In normal code this is illegal and probably an error, but when used like this in the define, it means the preprocessor token DEFINE should be used like a macro:

+ // if not even "()" is found [in the definition] then it's a normal preprocessor define, so handle as such+ if (tk->m_Args.IsEmpty())+ {+ expandedText = tk->m_FullType;+ return true; // return true for ReplaceBufferText()+ }+

// show calltip for __glewMultiDrawElements(...), which can be // a function, macro, function ptr or typedef'd function ptr// (currently function ptr are not parsed properly in DoParse() so // it's not supported for now. But typedef'd func ptrs are supported)glMultiDrawElements(

Likewise we have function calls and function names, see some example code:

bool Tokenizer::ReplaceFunctionLikeMacro(const Token* tk, bool updatePeekToken){ wxString macroExpandedText; if ( GetMacroExpendedText(tk, macroExpandedText) ) return ReplaceBufferText(macroExpandedText, updatePeekToken); return false;}But as I see that the function ReplaceFunctionLikeMacro is called in some condition that tk->m_Args is not empty. It looks like m_Args is used to distinguish whether a macro definition is variable-like or function-like, but if I realize that even function-like macro definitions are allowed to use empty formal argument list, so the condition is not true any more.

GetMacroExpendedText did some tricks, it just put the formal arguments before actual arguments, so the buffer becomes

Oh, so for a function-like macro definition which has empty argument list, the m_Args is "()". SplitArguments(actualArgs) function will still return true, but both the actualArgs and the formalArgs are empty, in this case, we can directly return the macro's definition string without any replacement.

BTW: it looks like ReplaceFunctionLikeMacro() only runs one level replacement, if you have several replacement rules (A -> B ->C), call this function only make a (A->B), but I think (B->C) will be happens after some GetToken() call. So, it looks like our macro expansion is not similar as the C language standard.

Quote

BTW: valid preprocessor tokens will still be expanded (i.e., ordinary preprocessor tokens that have no parentheses in the definition itself, like #define DEFINE ...I added that feature too in the patch:

+ // if not even "()" is found [in the definition] then it's a normal preprocessor define, so handle as such+ if (tk->m_Args.IsEmpty())+ {+ expandedText = tk->m_FullType;+ return true; // return true for ReplaceBufferText()+ }+

Yes, this is some kind of expanding variable-like macro.

Edit: I think I still need time to review the CC code and your patches.

« Last Edit: September 08, 2014, 08:25:21 am by ollydbg »

Logged

If some piece of memory should be reused, turn them to variables (or const variables).If some piece of operations should be reused, turn them to functions.If they happened together, then turn them to classes.

Nornally, when parsed, you get a variable Token named "AAA", since CC don't check every token for macros, so AAA does not trigger macro replacement.

If you create a user replacement rule by adding AAA -> @ in the CC's setting(this just move back the m_TokenIndex, and put the Tokenizer in macro replacement mode), you will get a variable Token named "EEE", which is correct.

I see that DoGetToken() can be recurively called, thus recursive macro replacement happens when DoGetToken() is called.EDIT:In your patch

//NOTE: ReplaceFunctionLikeMacro will recursively expand all defines and macro calls, // but will not expand macro names, which is what we want smallTokenizer.ReplaceFunctionLikeMacro(tk);I think the comment is not correct right? Since ReplacefunctionLikeMacro just do text replacement once.

if (m_RepeatReplaceCount > 0) { if (m_RepeatReplaceCount >= s_MaxRepeatReplaceCount) { m_TokenIndex = m_BufferLen - m_FirstRemainingLength; m_PeekAvailable = false;To solve this issue, I think1, collecting the used macros, and don't use it again2, expand all the macros once, not by recursive call of DoGetToken

« Last Edit: September 08, 2014, 10:34:19 am by ollydbg »

Logged

If some piece of memory should be reused, turn them to variables (or const variables).If some piece of operations should be reused, turn them to functions.If they happened together, then turn them to classes.

//NOTE: ReplaceFunctionLikeMacro will recursively expand all defines and macro calls, // but will not expand macro names, which is what we want smallTokenizer.ReplaceFunctionLikeMacro(tk);

I see that when you call ReplaceFunctionLikeMacro(), then the Tokenizer goes to the macro replacement mode, so the next time you call smallTokenizer.GetToken(), macro replacement will recursively expand all.

Logged

If some piece of memory should be reused, turn them to variables (or const variables).If some piece of operations should be reused, turn them to functions.If they happened together, then turn them to classes.

Oh, sorry about the confusion. Regarding macro expansion: for any macro, we have 1) the macro definition, and 2) the macro usage. Each of them can be of two kinds: 1) having parentheses (...), or 2) not having parentheses. So you can see there are totally 4 cases to handle. Then the macro can either be simple, or nested (needs recursive expansion). I will try to explain the different cases one by one.

// OPER(1) is an example of Case 3 macro usageint a = OPER(1);Here, OPER is defined as a variable-macro, but used as a function-macro. We should expand it like a variable-macro using plain ReplaceBufferText(). In my patch I have added support for this case (the test added at the beginning of GetMacroExpendedText()).

Case 4: Macro definition has the () but macro usage does not have it. Eg,

// MUL is an example of Case 4 macro usage#define OPER MUL // this line is valid (but CC should not expand MUL)

// these two lines are validint a = OPER(1);int b = MUL(1);

int c = MUL; // this line is invalid (CC should not expand MUL)Here, MUL is defined as a function-like macro. But in the line "#define OPER MUL" it is used without parentheses. This means our CC engine should treat it like an error and leave it alone (do not expand it). We can use this behavior to stop our recursive expansion at this point, and use the final result "MUL" for the calltips.

Quote from: ollydbg

It looks like this code above is valid, and I just create a simple test code, and it build fine under MinGW G++[...]

Yes, this is valid, but still we should not expand it (we need to stop there and use the MUL token for our calltips). Imagine if we expand it:

I see that the only usage of the GetMacroExpendedText() is here: (BTW: typo here? should be "expanded text"?)

Yes, here is the only usage of this function. GetMacroExpendedText() should return "true" if we want ReplaceBufferText(), and "false" if we want to leave the macro alone and don't replace. But even if there is empty () in the macro definition we have to handle it in GetMacroExpendedText() before returning true.And yes, it should be "expanded text".

Quote from: ollydbg

It looks like m_Args is used to distinguish whether a macro definition is variable-like or function-like, but if I realize that even function-like macro definitions are allowed to use empty formal argument list, so the condition is not true any more.

Yes, but m_Args is not empty even if there is empty parentheses. So we can safely check for m_Args->IsEmpty(). A function-macro like this: "#define MACRO() ..." will have in m_Args: "()". Only for variable-macro the m_Args is really empty.

Quote from: ollydbg

BTW: it looks like ReplaceFunctionLikeMacro() only runs one level replacement, if you have several replacement rules (A -> B ->C), call this function only make a (A->B), but I think (B->C) will be happens after some GetToken() call.

Quote from: ollydbg

I think the comment is not correct right? Since ReplacefunctionLikeMacro just do text replacement once.

Well, yes, the actual recursive expansion happens in the next GetToken() call, but I wanted to keep the comments simple and easy to read. Technically, ReplacefunctionLikeMacro() expands one level, and turns on the flag for further expansion in the next GetToken() or PeekToken(). In my patch there is:

+ smallTokenizer.ReplaceFunctionLikeMacro(tk);+ tk = tree->at(tree->TokenExists(smallTokenizer.GetToken(), ...So, full recursive expansion occurs in the GetToken(), but when we hit Case 4, GetMacroExpendedText() will return false, and it won't be expanded further. We use that result for the calltips.