Re[2]: [CEDET-devel] Some proposals for Semantic 2.0

>>> David Ponce <david@...> seems to think that:
>Hi Eric,
>
>[...]
> > This patch looks good. Is there a reason that `semantic-init' and
> > `semantic-lex' was moved lower? I could not divine that.
>
>In fact only `semantic-lex' needed to be moved lower to satisfy
>`defsubst' dependencies. For consistency I moved `semantic-lex-init'
>too.
>
>I checked it in plus Wisent's changes ;-)
Aha, thanks.
> > > My second point is the following proposal for an enhanced API to
> > > manage nonterminal expansion (see the code at end).
>[...]
>
> > This is an interesting idea. On one side, we save a function call.
> > On the other, you cannot write an expander that does the same thing
> > to two different types of tokens without adding more layers of
> > indirection.
> >
> > Of course, I'm not sure when you would want to do that, but you
> > never know. I think the simplicity this adds outweighs a possible
> > future need.
>
>I investigated a little more on this new mechanism...
>
>I did performance measurements (Emacs 21.3.50.1 & Emacs 20.7.3) with
>the current version and the new one, and surprisingly not noticed any
>ameliorations! In most cases the current implementation performs a
^ Cool word.
>little faster than the new one! So, for now, I don't plan to work
>more on that.
[ ... ]
Makes sense. Thanks for clever hack.
Eric
--
Eric Ludlam: zappo@..., eric@...
Home: http://www.ultranet.com/~zappo Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net GNU: http://www.gnu.org

Thread view

Hi Eric,
As I am currently on vacation, I got some free time to work on
Semantic 2.0 :-)
First, I submit you a patch to semantic-lex.el and semantic.el to
simplify and improve some code. See the attached file semantic.patch
for details.
My second point is the following proposal for an enhanced API to
manage nonterminal expansion (see the code at end).
* The new API can work like the old one.
In mode hook, instead of using:
(setq semantic-expand-nonterminal 'my-expander)
you must write:
(semantic-install-nonterminal-expanders
'((t . my-expander)))
* To avoid unnecessary call to the expander on every tokens (and
save CPU!), it is now also possible to do something like this:
(semantic-install-nonterminal-expanders
'((variable . my-expander)
(type . my-expander)))
* Even better it is possible to use specific expanders, like this:
(semantic-install-nonterminal-expanders
'((variable . my-variable-expander)
(type . my-type-expander)))
What do you think?
Sincerely,
David
--------------- The code:
(defvar semantic-expand-nonterminal-obarray [0]
"Buffer local obarray of nonterminal expanders.
Use the function `semantic-install-nonterminal-expanders' to install
expanders.")
(make-variable-buffer-local 'semantic-expand-nonterminal-obarray)
(defun semantic-install-nonterminal-expanders (expanders)
"Install nonterminal EXPANDERS in `semantic-nonterminal-expanders'.
EXPANDERS must be an alist of elements:
(TOKEN-CLASS . FUNCTION)
TOKEN-CLASS is a symbol equal to a semantic token class to expand
\(like type, function, variable, etc.), or t for all other ones.
FUNCTION is the function to expand the token with. It will receive
the semantic token to expand."
(if (eq semantic-expand-nonterminal-obarray
(default-value 'semantic-expand-nonterminal-obarray))
(setq semantic-expand-nonterminal-obarray (make-vector 13 0)))
(while expanders
(set (intern (symbol-name (caar expanders))
semantic-expand-nonterminal-obarray)
(cdar expanders))
(setq expanders (cdr expanders)))
semantic-expand-nonterminal-obarray)
(defsubst semantic-nonterminal-expander (token)
"Find and return the expander function for TOKEN.
Return nil if not found."
(symbol-value
(or (intern-soft (symbol-name (semantic-token-token token))
semantic-expand-nonterminal-obarray)
(intern-soft "t" semantic-expand-nonterminal-obarray))))
;; Following is the updated version of `semantic-raw-to-cooked-token'
;; to use the new mechanism.
(defun semantic-raw-to-cooked-token (token)
"Convert TOKEN from a raw state to a cooked state.
The parser returns raw tokens with positional data START/END. We
convert it from that to a cooked state with a property list and a
vector [START END]. The raw token is changed with side effects and
maybe expanded in several cooked tokens when the variable
`semantic-expand-nonterminal' is set. So this function always returns
a list of cooked tokens."
;; Because some parsers can return tokens already cooked (wisent is
;; an example), check if TOKEN was already cooked to just return it.
(if (semantic-cooked-token-p token)
token
(let* ((ncdr (- (length token) 2))
(propcdr (if (natnump ncdr) (nthcdr ncdr token)))
(rngecdr (cdr propcdr))
;; propcdr is the CDR containing the START from the token.
;; rngecdr is the CDR containing the END from the token.
;; PROPCDR will contain the property list after cooking.
;; RNGECDR will contain the [START END] vector after cooking.
(range (condition-case nil
(vector (car propcdr) (car rngecdr))
(error (debug token)
nil)))
result expander expandedtokens)
;; Convert START/END into PROPERTIES/[START END].
(setcar rngecdr range)
(setcar propcdr nil)
;; Expand based on local configuration
(if (setq expander (semantic-nonterminal-expander token))
;; Glom generated tokens. THESE TOKENS MUST BE VALID ONES!
(setq expandedtokens (funcall expander token)
result (if expandedtokens
(append expandedtokens result)
(cons token result)))
;; No expanders
(setq result (cons token result)))
result)))

>>> David Ponce <david@...> seems to think that:
>Hi Eric,
>
>As I am currently on vacation, I got some free time to work on
>Semantic 2.0 :-)
>
>First, I submit you a patch to semantic-lex.el and semantic.el to
>simplify and improve some code. See the attached file semantic.patch
>for details.
This patch looks good. Is there a reason that `semantic-init' and
`semantic-lex' was moved lower? I could not divine that.
>My second point is the following proposal for an enhanced API to
>manage nonterminal expansion (see the code at end).
>
>* The new API can work like the old one.
>
> In mode hook, instead of using:
>
> (setq semantic-expand-nonterminal 'my-expander)
>
> you must write:
>
> (semantic-install-nonterminal-expanders
> '((t . my-expander)))
>
>* To avoid unnecessary call to the expander on every tokens (and
> save CPU!), it is now also possible to do something like this:
>
> (semantic-install-nonterminal-expanders
> '((variable . my-expander)
> (type . my-expander)))
>
>* Even better it is possible to use specific expanders, like this:
>
> (semantic-install-nonterminal-expanders
> '((variable . my-variable-expander)
> (type . my-type-expander)))
>
>What do you think?
This is an interesting idea. On one side, we save a function call.
On the other, you cannot write an expander that does the same thing
to two different types of tokens without adding more layers of
indirection.
Of course, I'm not sure when you would want to do that, but you never
know. I think the simplicity this adds outweighs a possible future
need.
Eric
--
Eric Ludlam: zappo@..., eric@...
Home: http://www.ultranet.com/~zappo Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net GNU: http://www.gnu.org

Hi Eric,
[...]
> This patch looks good. Is there a reason that `semantic-init' and
> `semantic-lex' was moved lower? I could not divine that.
In fact only `semantic-lex' needed to be moved lower to satisfy
`defsubst' dependencies. For consistency I moved `semantic-lex-init'
too.
I checked it in plus Wisent's changes ;-)
> > My second point is the following proposal for an enhanced API to
> > manage nonterminal expansion (see the code at end).
[...]
> This is an interesting idea. On one side, we save a function call.
> On the other, you cannot write an expander that does the same thing
> to two different types of tokens without adding more layers of
> indirection.
>
> Of course, I'm not sure when you would want to do that, but you
> never know. I think the simplicity this adds outweighs a possible
> future need.
I investigated a little more on this new mechanism...
I did performance measurements (Emacs 21.3.50.1 & Emacs 20.7.3) with
the current version and the new one, and surprisingly not noticed any
ameliorations! In most cases the current implementation performs a
little faster than the new one! So, for now, I don't plan to work
more on that.
Thanks for your time!
David

>>> David Ponce <david@...> seems to think that:
>Hi Eric,
>
>[...]
> > This patch looks good. Is there a reason that `semantic-init' and
> > `semantic-lex' was moved lower? I could not divine that.
>
>In fact only `semantic-lex' needed to be moved lower to satisfy
>`defsubst' dependencies. For consistency I moved `semantic-lex-init'
>too.
>
>I checked it in plus Wisent's changes ;-)
Aha, thanks.
> > > My second point is the following proposal for an enhanced API to
> > > manage nonterminal expansion (see the code at end).
>[...]
>
> > This is an interesting idea. On one side, we save a function call.
> > On the other, you cannot write an expander that does the same thing
> > to two different types of tokens without adding more layers of
> > indirection.
> >
> > Of course, I'm not sure when you would want to do that, but you
> > never know. I think the simplicity this adds outweighs a possible
> > future need.
>
>I investigated a little more on this new mechanism...
>
>I did performance measurements (Emacs 21.3.50.1 & Emacs 20.7.3) with
>the current version and the new one, and surprisingly not noticed any
>ameliorations! In most cases the current implementation performs a
^ Cool word.
>little faster than the new one! So, for now, I don't plan to work
>more on that.
[ ... ]
Makes sense. Thanks for clever hack.
Eric
--
Eric Ludlam: zappo@..., eric@...
Home: http://www.ultranet.com/~zappo Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net GNU: http://www.gnu.org

Community

Help

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

I agree to receive quotes, newsletters and other information from sourceforge.net and its partners regarding IT services and products. I understand that I can withdraw my consent at any time. Please refer to our Privacy Policy or Contact Us for more details