% GenI surface realiser
% Copyright (C) 2005 Carlos Areces and Eric Kow
%
% This program is free software; you can redistribute it and/or
% modify it under the terms of the GNU General Public License
% as published by the Free Software Foundation; either version 2
% of the License, or (at your option) any later version.
%
% This program is distributed in the hope that it will be useful,
% but WITHOUT ANY WARRANTY; without even the implied warranty of
% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
% GNU General Public License for more details.
%
% You should have received a copy of the GNU General Public License
% along with this program; if not, write to the Free Software
% Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
\chapter{SimpleBuilder}
\label{cha:SimpleBuilder}
A SimpleBuilder is a Builder which constructs derived trees using a
simple agenda control mechanism and two-phase realisation (substitution
before adjunction). There is no packing strategy whatsoever; each chart
item is a derived tree.
\begin{code}

\end{code}
\subsection{SimpleState and SimpleStatus}
The \fnreflite{SimpleState} is a state monad where the state being
thread through is a \fnreflite{SimpleStatus}. The two are named
deliberately alike to indicate their close relationship.
To prevent confusion, we ought to keep a somewhat consistent naming
scheme across the builders: FooState for the state monad, FooStatus for
the state monad's ``contents'', and FooItem for the chart items
manipulated.
Note the theTrash is not actually essential to the operation of the
generator; it is for pratical debugging of grammars. Instead of
trees dissapearing off the face of the debugger; they go into the
trash where the user can inspect them and try to figure out why they
went wrong.
\begin{code}

typeSimpleStatea=B.BuilderStateSimpleStatusadataSimpleStatus=S{theAgenda::Agenda,theAuxAgenda::AuxAgenda,theChart::Chart#ifndef DISABLE_GUI,theTrash::Trash#endif,theResults::[SimpleItem],theIafMap::IafMap-- for index accessibility filtering,tsem::BitVector,step::Ptype,gencounter::Integer,genconfig::Params-- we keep a SemBitMap strictly to help display the semantics,semBitMap::SemBitMap}derivingShow

\end{code}
\subsubsection{SimpleStatus updaters}
\begin{code}

addToAgenda::SimpleItem->SimpleState()addToAgendate=domodify$\s->s{theAgenda=te:(theAgendas)}updateAgenda::Agenda->SimpleState()updateAgendaa=domodify$\s->s{theAgenda=a}addToAuxAgenda::SimpleItem->SimpleState()addToAuxAgendate=dos<-get-- each new tree gets a unique id... this makes comparisons fasterletcounter=(gencounters)+1te2=te{siId=counter}puts{gencounter=counter,theAuxAgenda=te2:(theAuxAgendas)}addToChart::SimpleItem->SimpleState()addToChartte=domodify$\s->s{theChart=te:(theCharts)}incrCounterchart_size1#ifndef DISABLE_GUIaddToTrash::SimpleItem->String->SimpleState()addToTrashteerr=dolette2=modifyGuiStuff(\g->g{siDiagnostic=err:(siDiagnosticg)})temodify$\s->s{theTrash=te2:(theTrashs)}#endifaddToResults::SimpleItem->SimpleState()addToResultste=domodify$\s->s{theResults=te:(theResultss)}

\end{code}
\subsection{SimpleItem}
\begin{code}

dataSimpleItem=SimpleItem{siId::ChartId--,siSubstnodes::![TagSite],siAdjnodes::![TagSite]--,siSemantics::!BitVector,siPolpaths::!BitVector-- for generation sans semantics-- , siAdjlist :: [(String,Integer)] -- (node name, auxiliary tree id)-- for index accesibility filtering (one-phase only),siAccesible::[String]-- it's acc/inacc/undetermined,siInaccessible::[String]-- that's why you want both---- | actually: a set of pre-terminals and their leaves,siLeaves::[(String,B.UninflectedDisjunction)],siDerived::TreeString,siRoot::TagSite,siFoot::MaybeTagSite--,siPendingTb::[TagSite]-- only for one-phase-- how was this item produced?,siDerivation::TagDerivation#ifndef DISABLE_GUI-- for the debugger only,siGuiStuff::SimpleGuiItem#endif}derivingShow#ifndef DISABLE_GUI-- | Things whose only use is within the graphical debuggerdataSimpleGuiItem=SimpleGuiItem{siHighlight::[String]-- ^ nodes to highlight,siNodes::[GNode]-- ^ actually a set-- if there are things wrong with this item, what?,siDiagnostic::[String],siFullSem::Sem,siIdname::String}derivingShowmodifyGuiStuff::(SimpleGuiItem->SimpleGuiItem)->SimpleItem->SimpleItemmodifyGuiStufffni=i{siGuiStuff=fn.siGuiStuff$i}#endiftypeChartId=IntegerinstanceReplacableSimpleItemwherereplaceMapsi=i{siSubstnodes=replaceMaps(siSubstnodesi),siAdjnodes=replaceMaps(siAdjnodesi),siLeaves=replaceMaps(siLeavesi),siRoot=replaceMaps(siRooti),siFoot=replaceMaps(siFooti),siPendingTb=replaceMaps(siPendingTbi)#ifndef DISABLE_GUI,siGuiStuff=replaceMaps(siGuiStuffi)#endif}replaceOne=replaceOneAsMap#ifndef DISABLE_GUIinstanceReplacableSimpleGuiItemwherereplaceMapsi=i{siNodes=replaceMaps(siNodesi)}replaceOne=replaceOneAsMap#endif

\end{code}
\begin{code}

{-# INLINE closedAux #-}-- | True if the chart item has no open substitution nodesclosed::SimpleItem->Boolclosed=null.siSubstnodes-- | True if the chart item is an auxiliary treeaux::SimpleItem->Boolaux=isJust.siFoot-- | True if both 'closed' and 'aux' are TrueclosedAux::SimpleItem->BoolclosedAuxx=(auxx)&&(closedx)adjdone::SimpleItem->Booladjdone=null.siAdjnodessiInitial::SimpleItem->BoolsiInitial=isNothing.siFoot

\end{code}
% --------------------------------------------------------------------
\section{Generate}
% --------------------------------------------------------------------
\subsection{One-phase generation}
This is a standard chart-and-agenda mechanism, where each iteration
consists of picking an item off the agenda and combining it with
elements from the chart.
\begin{code}

generateStep_1p::SimpleState()generateStep_1p=doisDone<-gets(null.theAgenda)iaf<-gets(isIaf.genconfig)letdispatch=mapM(simpleDispatch_1piaf)ifisDonethenreturn()elsedoincrCounternum_iterations1given<-selectGiven-- do both substitution and adjunctionapplySubstitution1pgiven>>=dispatchpassiveAdjunction1pgiven>>=dispatchactiveAdjunction1pgiven>>=dispatchsansAdjunction1pgiven>>=dispatch-- determine which of the res should go in the agenda-- (monadic state) and which should go in the result (res')addToChartgiven

\end{code}
\subsection{Two-phase generation}
Following \cite{carroll1999ecg}, we could also separate realisation into
two distinct phases. This requires that we maintain two seperate
agendas and process them sequentially, one loop after the other. See
\fnref{switchToAux} for details.
\begin{itemize}
\item If both Agenda and AuxAgenda are empty then there is nothing to do,
otherwise, if Agenda is empty then we switch to the application of the
Adjunction rule.
\item After the rule is applied we classify solutions into those that are complete
and cover the semantics and those that don't. The first ones are returned
and added to the result, while the others are sent back to Agenda.
\item Notice that if we are applying the Substitution rule then the
current agenda item is added to the chart, otherwise it is deleted.
\end{itemize}
\begin{code}

generateStep_2p::SimpleState()generateStep_2p=donir<-gets(null.theAgenda)curStep<-getsstep-- this check may seem redundant with generate, but it's needed-- to protect against a user who calls generateStep_2p on a finished-- stateif(nir&&curStep==Auxiliar)thenreturn()elsedoincrCounternum_iterations1-- this triggers exactly once in the whole processifnirthenswitchToAuxelsegenerateStep_2p'generateStep_2p'::SimpleState()generateStep_2p'=do-- choose an item from the agendagiven<-selectGiven-- have we triggered the switch to aux yet?curStep<-getsstep-- do either substitution or adjunctionres<-if(curStep==Initial)thenapplySubstitutiongivenelseliftM2(++)(sansAdjunction2pgiven)(applyAdjunction2pgiven)-- determine which of the res should go in the agenda-- (monadic state) and which should go in the result (res')mapMsimpleDispatch_2pres-- put the given into the chart untouchedif(curStep==Initial)thenaddToChartgivenelsewhen(adjdonegiven)$trashItgiven

\end{code}
\subsection{Switching phases}
\fnlabel{switchToAux} When all substitutions has been done, tags with
substitution nodes still open are deleted, then the auxiliars tags are put in
Chart and the (initial) tags in the repository are moved into the Agenda. The
step is then changed to Auxiliary
\begin{code}

switchToAux::SimpleState()switchToAux=dost<-getletchart=theChartstconfig=genconfigst-- You might be wondering why we ignore the auxiliary trees in the-- chart; this is because all the syntactically complete auxiliary-- trees have already been filtered away by calls to classifyNewinitialT=filtersiInitialchartres1@(compT1,incompT1)=partition(null.siSubstnodes)initialT--auxAgenda=theAuxAgendast(compT2,incompT2)=ifsemfilteredconfigthensemfilter(tsemst)auxAgendacompT1elseres1--compT=compT2putst{theAgenda=[],theAuxAgenda=[],theChart=auxAgenda,step=Auxiliar}-- the root cat filter by ClaireletswitchFilter=ifrootcatfilteredconfigthendpRootFeatFailure2>-->dpToAgendaelsedpToAgendamapMswitchFiltercompT-- toss the syntactically incomplete stuff in the trash#ifndef DISABLE_GUImapM(\t->addToTrashtts_synIncomplete)incompT1mapM(\t->addToTrasht"sem-filtered")incompT2#endifreturn()

\end{code}
\subsubsection{SemFilter Optimisation}
\label{sec:semfilter}
The purpose of the semantic filter optimisation is to take full
advantage of Carroll's delayed adjunction. Consider the semantics
\semexpr{def(m), poor(m), brokenhearted(m), man(m), def(w), woman(w),
beautiful(w), heartless(w), rejects(w,m)}. At the switchToAux step, we
are left with the initial trees \natlang{man}, \natlang{woman}, \natlang{the
woman rejects the man}.
It would be nice to filter out the structures \natlang{man} and \natlang{woman}
since we know that they are not going to be semantically complete even with
adjunction. More precisely, on the switch to adjunction, we do the following:
\begin{itemize}
\item Take the union of the semantics of all auxiliary trees; which
we call $\phi^*$
\item Delete any initial tree with semantics $\phi^s$ such that
$\phi^s \cup \phi^*$ is not the target semantics
\end{itemize}
In other words, we delete all initial trees that cannot produce a semantically
complete result even with the help of auxiliary trees.
FIXME: comment 2006-04-18: sem filter each polarity path separately (this is
more aggressive; it gives us much more filtering)
\begin{code}

semfilter::BitVector->[SimpleItem]->[SimpleItem]->([SimpleItem],[SimpleItem])semfilterinputsemauxsinitial=letauxsemx=foldl'(.|.)0[siSemanticsa|a<-auxs,siPolpathsa.&.siPolpathsx/=0]-- lite, here, means sans auxiliary semanticsnotjunkx=(siSemanticsx).&.inputsemLite==inputsemLitewhereinputsemLite=inputsem`xor`(auxsemx)-- note that we can't just compare against siSemantics because-- that would exclude trees that have stuff in the aux semantics-- which would be overzealousinpartitionnotjunkinitial

\end{code}
% --------------------------------------------------------------------
\section{Operations}
% --------------------------------------------------------------------
We implement the two TAG operations, substitution and adjunction, below.
These are the only two operations we have, because we're working with a
very simple builder that constructs derived trees.
% --------------------------------------------------------------------
\subsection{Substitution}
\label{sec:substitution}
% --------------------------------------------------------------------
\paragraph{applySubstitution} Given a SimpleItem it returns the list of all
possible substitutions between it and the elements in Chart
\begin{code}

applySubstitution::SimpleItem->SimpleState([SimpleItem])applySubstitutionitem=dogr<-lookupChartitemactive<-mapM(\x->iapplySubstTrueitemx)grpassive<-mapM(\x->iapplySubstTruexitem)grletres=concat$active++passiveincrCounternum_comparisons(2*(lengthgr))returnresapplySubstitution1p::SimpleItem->SimpleState([SimpleItem])applySubstitution1pitem=dogr<-lookupChartitemactive<-ifadjdoneitemthenreturn[]elsemapM(\x->iapplySubstFalseitemx)grpassive<-mapM(\x->iapplySubstFalsexitem)$filteradjdonegrletres=concat$active++passiveincrCounternum_comparisons(2*(lengthgr))returnres-- | Note: returns ONE possible substitution (the head node)-- of the first in the second. As all substitutions nodes should-- be substituted we force substitution in order.iapplySubst::Bool->SimpleItem->SimpleItem->SimpleState[SimpleItem]iapplySubsttwophaseitem1item2|siInitialitem1&&closeditem1={-# SCC "applySubstitution" #-}casesiSubstnodesitem2of[]->return[]((TagSitenfufdnOrigin):stail)->letdoIt=do-- Maybe monadletr@(TagSiternrurdrOrigin)=siRootitem1(newU,subst1)<-unifyFeatrufu(newD,subst2)<-unifyFeat(replacesubst1rd)(replacesubst1fd)letsubst=mergeSubstsubst1subst2nr=TagSiternnewUnewDrOriginadj1=nr:(deleter$siAdjnodesitem1)adj2=siAdjnodesitem2#ifdef DISABLE_GUIitem1g=item1#elseitem1g=item1{siGuiStuff=g2}whereg2=g{siNodes=repList(gnnameIsrn)newRoot(siNodesg)}g=siGuiStuffitem1-- gui stuffnewRootg=g{gup=newU,gdown=newD,gtype=Other}#endifletpending=iftwophasethen[]elsenr:((siPendingTbitem1)++(siPendingTbitem2))return$!replacesubst$combineSimpleItems[rn]item1g$item2{siSubstnodes=stail++(siSubstnodesitem1),siAdjnodes=adj2++adj1,siDerived=plugTree(siDeriveditem1)n(siDeriveditem2),siDerivation=addToDerivation's'(item1g,rOrigin)(item2,nOrigin,n),siLeaves=(siLeavesitem1)++(siLeavesitem2),siPendingTb=pending}incasedoItofNothing->return[]Justx->doincrCounter"substitutions"1return[x]iapplySubst___=return[]

\end{code}
% --------------------------------------------------------------------
\subsection{Adjunction}
\label{sec:adjunction}
\label{sec:ordered_adjunction}
\label{sec:foot_constraint}
% ---------------------------------------------------------------
\paragraph{applyAdjunction2p} Given a SimpleItem, it returns the list of all
possible adjunctions between it and the elements in Chart.
The Chart contains Auxiliars, while SimpleItem is an Initial
Note: as of 13 april 2005 - only uses ordered adjunction as described in
\cite{kow04a}
\begin{code}

\end{code}
Note that in the one-phase variant of non-adjunction, we can't do top/bot
unification on the fly, because afaik we can't tell that a node will never
be revisited. One example of this is if you try to adjoin into the root
\begin{code}

-- | Ignore the next adjunction nodesansAdjunction1p,sansAdjunction2p::SimpleItem->SimpleState[SimpleItem]sansAdjunction1pitem|closeditem=casesiAdjnodesitemof[]->return[](ahead:atail)->return$[item{siAdjnodes=atail,siPendingTb=ahead:(siPendingTbitem)}]sansAdjunction1p_=return[]-- | Ignore the next adjunction nodesansAdjunction2pitem|closeditem=casesiAdjnodesitemof[]->return[](TagSitegntbo:atail)->do-- do top/bottom unification on the nodecaseunifyFeattbofNothing->#ifndef DISABLE_GUIdoaddToTrash(modifyGuiStuff(\g->g{siHighlight=[gn]})item)ts_tbUnificationFailure#endifreturn[]Just(tb,s)->letitem1=ifisRootOfitemgnthenitem{siRoot=TagSitegntb[]o}elseitem#ifdef DISABLE_GUIitem2=item1#elseitem2=modifyGuiStuff(constrainAdjgntb)item1#endifinreturn$![replaces$!item2{siAdjnodes=atail}]sansAdjunction2p_=return[]

\end{code}
The main work for adjunction is done in the helper function below
(see also figure \ref{fig:adjunction}).
Auxiliary tree \texttt{te1} has a root node \texttt{r} and a foot
node \texttt{f}. Main tree \texttt{te2} has an adjunction site \texttt{an}.
The resulting tree \texttt{res} is a result of splicing \texttt{te1} into
\texttt{te2}. We replace \texttt{s} with the nodes \texttt{anr} and
\texttt{anf} (which are the results of unifying \texttt{an} with \texttt{r}
and \texttt{f} respectively).
\begin{figure}
\begin{center}
\includegraphics[scale=0.5]{images/adjunction.pdf}
\label{fig:adjunction}
\caption{iapplyAdjNode}
\end{center}
\end{figure}
In addition to the trees proper, we have to consider that each tree has
a list with a copy of its adjunction sites. The adjunction list of the
result (\texttt{adjnodes res}) should then contain \texttt{adjnodes te1}
and \texttt{adjnodes te2}, but replacing \texttt{r} and \texttt{an}
with \texttt{anr}.
\begin{code}

iapplyAdjNode::Bool->SimpleItem->SimpleItem->MaybeSimpleItemiapplyAdjNodetwophaseaItempItem={-# SCC "iapplyAdjNode" #-}casesiAdjnodespItemof[]->Nothing(TagSitean_namean_upan_downnOrigin:atail)->do-- block repeated adjunctions of the same SimpleItem (for ignore semantics mode)-- guard $ not $ (an_name, siId aItem) `elem` (siAdjlist pItem)-- let's go!letr@(TagSiter_namer_upr_downrOrigin)=siRootaItem-- auxiliary tree, eh?(TagSitef_namef_upf_down_)<-siFootaItem-- should really be an error if fails(anr_up',subst1)<-unifyFeatr_upan_up(anf_down,subst2)<-unifyFeat(replacesubst1f_down)(replacesubst1an_down)let-- combined substitution list and success conditionsubst12=mergeSubstsubst1subst2-- the result of unifying the t1 root and the t2 ananr=TagSiter_name(replacesubst2anr_up')r_downrOriginletanf_up=replacesubst12f_up-- the new adjunction nodesauxlite=deleter$siAdjnodesaItemnewadjnodes=anr:(atail++auxlite)--#ifdef DISABLE_GUIaItem2=aItem#else-- Ugh, this is horrible: this is just to make sure the GUI gets-- updated accordingly. The code used to be a lot simpler, but-- I started trying to move stuff out of the way in the interests-- of efficiency, and to pack as much gui-related stuff as possible-- into a single tuple.aItem2=aItem{siGuiStuff=fixNodes$siGuiStuffaItem}wherefixNodesg=g{siNodes=map(setSitesanr)(siNodesg)}setSites(TagSitenud_)gn=ifgnnamegn==nthengn{gup=u,gdown=d}elsegn#endifrawCombined=combineSimpleItems[r_name,an_name]aItem2$pItem{siAdjnodes=newadjnodes,siLeaves=siLeavesaItem++siLeavespItem,siDerived=spliceTreef_name(siDerivedaItem)an_name(siDerivedpItem),siDerivation=addToDerivation'a'(aItem,rOrigin)(pItem,nOrigin,an_name)-- , siAdjlist = (n, (tidnum te1)):(siAdjlist item2)-- if we adjoin into the root, the new root is that of the aux-- tree (affects 1p only),siRoot=ifisRootOfpIteman_namethenrelsesiRootpItem,siPendingTb=iftwophasethen[]else(TagSitean_nameanf_upanf_downnOrigin):(siPendingTbpItem)++(siPendingTbaItem)}-- one phase = postpone tb unification-- two phase = do tb unification on the flyfinalRes1p=return$replacesubst12rawCombinedfinalRes2p=do-- tb on the former foottbRes<-unifyFeatanf_upanf_down#ifdef DISABLE_GUIlet(_,subst3)=tbResmyRes=res'#elselet(anf_tb,subst3)=tbResmyRes=modifyGuiStuff(constrainAdjan_nameanf_tb)res'#endif-- apply the substitutionsres'=replace(mergeSubstsubst12subst3)rawCombinedreturnmyRes-- ---------------iftwophasethenfinalRes2pelsefinalRes1p

ncopy::GNode->TagSitencopyx=TagSite(gnnamex)(gupx)(gdownx)(goriginx)isRootOf::SimpleItem->String->BoolisRootOfitemn=n==rnamewhere(TagSitername___)=siRootitem-- | Retrieves a list of trees from the chart which could be combined with the given agenda tree.-- The current implementation searches for trees which-- * do not have overlapping semantics with the given-- * are on the some of the same polarity automaton paths as the-- current agenda itemlookupChart::SimpleItem->SimpleState[SimpleItem]lookupChartgiven=dochart<-getstheChartletgpaths=siPolpathsgivengsem=siSemanticsgivenreturn[i|i<-chart-- should be on the same polarity path (chart sharing),(siPolpathsi).&.gpaths/=0-- semantics should not be overlapping&&(siSemanticsi.&.gsem)==0]-- | Helper function for when chart operations succeed.combineSimpleItems::[NodeName]-- ^ nodes to highlight->SimpleItem->SimpleItem->SimpleItemcombineSimpleItemshiitem1item2={-# SCC "combineSimpleItems" #-}item2{siSemantics=(siSemanticsitem1).|.(siSemanticsitem2),siPolpaths=(siPolpathsitem1).&.(siPolpathsitem2)#ifndef DISABLE_GUI,siGuiStuff=combineSimpleGuiItemshi(siGuiStuffitem1)(siGuiStuffitem2)#endif}#ifndef DISABLE_GUIcombineSimpleGuiItems::[NodeName]->SimpleGuiItem->SimpleGuiItem->SimpleGuiItemcombineSimpleGuiItemshiitem1item2=item2{siFullSem=sortSem$(siFullSemitem1)++(siFullSemitem2),siNodes=(siNodesitem1)++(siNodesitem2),siDiagnostic=(siDiagnosticitem1)++(siDiagnosticitem2),siHighlight=hi}constrainAdj::String->Flist->SimpleGuiItem->SimpleGuiItemconstrainAdjgnnewTg=g{siNodes=repList(gnnameIsgn)fixIt(siNodesg)}wherefixItn=n{gup=newT,gdown=[],gaconstr=True}#endif

\end{code}
\subsubsection{Derivation trees}
We make the simplifying assumption that each chart item is only used once.
This is clearly wrong if we allow for items with an empty semantics, but
since we do not actually allow such a thing, we're ok.
\begin{code}

\end{code}
% --------------------------------------------------------------------
\section{Dispatching new results}
% --------------------------------------------------------------------
Dispatching is the process where new chart items are assigned to one of
the trash, agenda, auxiliary agenda or chart. The item could be
modified during dispatch-time; for example, we might do top/bottom
unification on it. See \ref{sec:dispatching} for more details.
\begin{code}

typeSimpleDispatchFilter=DispatchFilterSimpleStateSimpleItemsimpleDispatch_2p::SimpleDispatchFiltersimpleDispatch_2p=simpleDispatch(dpRootFeatFailure>-->dpToResults)(dpAux>-->dpToAgenda)simpleDispatch_1p::Bool->SimpleDispatchFiltersimpleDispatch_1piaf=simpleDispatch(dpRootFeatFailure>-->dpTbFailure>-->dpToResults)(maybeDpIaf>-->dpToAgenda)wheremaybeDpIaf=ifiafthendpIafFailureelsenullFiltersimpleDispatch::SimpleDispatchFilter->SimpleDispatchFilter->SimpleDispatchFiltersimpleDispatchresFilternonResFilteritem=doinputsem<-getstsemletsynCompletex=siInitialx&&closedx&&adjdonexsemCompletex=inputsem==siSemanticsxisResultx=synCompletex&&semCompletexcondFilterisResultresFilternonResFilteritemdpAux,dpToAgenda::SimpleDispatchFilterdpTbFailure,dpRootFeatFailure,dpRootFeatFailure2,dpToResults::SimpleDispatchFilterdpToTrash::String->SimpleDispatchFilterdpToAgendax=addToAgendax>>returnNothingdpToResultsx=addToResultsx>>returnNothing#ifdef DISABLE_GUIdpToTrash__=returnNothing#elsedpToTrashmx=addToTrashxm>>returnNothing#endifdpAuxitem=ifclosedAuxitemthenaddToAuxAgendaitem>>returnNothingelsereturn$Justitem{-
-- | Dispatches to the trash and returns Nothing if there is a tree
-- size limit in effect and the item is over that limit. The
-- tree size limit is used in 'IgnoreSemantics' mode.
dpTreeLimit item =
do config <- gets genconfig
case maxTrees config of
Nothing -> return $ Just item
Just lim -> if (length.snd.siDerivation) item > lim
then do addToTrash item (ts_overnumTrees lim)
return Nothing
else return $ Just item
where ts_overnumTrees l = "Over derivation size of " ++ (show l)
-}-- | This is only used for the one-phase algorithmdpTbFailureitem=return$iftbUnifyTreeitemthenJustitemelseNothing-- | If the item (ostensibly a result) does not have the correct root-- category, return Nothing; otherwise return Just itemdpRootFeatFailure=dpRootFeatFailure_FalsedpRootFeatFailure2=dpRootFeatFailure_TruedpRootFeatFailure_::Bool->SimpleDispatchFilterdpRootFeatFailure_countitem=doconfig<-getsgenconfigletrootFeat=getListFlagPRootFeatureFlgconfig(TagSite_top__)=siRootitemcaseunifyFeatrootFeattopofNothing->dowhencount$incrCounter"root_feat_discards"1dpToTrash(ts_rootFeatureMismatchrootFeat)itemJust(_,s)->return.Just$replacesitem

\end{code}
% --------------------------------------------------------------------
\subsection{Top and bottom unification}
% --------------------------------------------------------------------
\paragraph{tbUnifyTree} unifies the top and bottom feature structures
of each node on each tree. Note: this only determines if it is
possible to do so. Actually returning the results is possible
and even easy
(you'll have to look back into the darcs repository and unpull the
patch from 2006-05-21T15:40:51 ``Remove top/bot unification standalone
code.'')
but since it is only used in the one-phase algorithm and for the
graphical interface, I decided not to bother.
\begin{code}

\end{code}
Our helper function corresponds to the first unification step. It is
meant to be called from a fold. The node argument represents the
current node being explored. The Either argument holds a list of
pending substitutions and a copy of the entire tree.
There are two things going on in here:
\begin{enumerate}
\item check if unification is possible - first we apply the pending
substitutions on the node and then we check if unification
of the top and bottom feature structures of that node
succeeds
\item keep track of the substitutions that need to be performed -
any new substitutions that result from unification are
added to the pending list
\end{enumerate}
Note that we wrap the second argument in a Maybe; this is used to
indicate that if unification suceeds or fails. We also use it to
prevent the function from doing any work if a unification failure
from a previous call has already occured.
Getting this right was a big pain in the butt, so don't go trying to
simplify this over-complicated code unless you know what you're doing.
\begin{code}

tbUnifyNode::TbEither->TagSite->TbEithertbUnifyNode(Rightpending)rawSite=-- apply pending substitutionscasereplacependingrawSiteof(TagSitenameupdown_)->-- check top/bottom unification on this nodecaseunifyFeatupdownof-- stop all future iterationsNothing->Leftname-- apply any new substutions to the whole treeJust(_,sb)->Right(mergeSubstpendingsb)-- if earlier we had a failure, don't even bothertbUnifyNode(Leftn)_=Leftn

\end{code}
% --------------------------------------------------------------------
\subsection{Index accesibility filtering}
\label{sec:simple:iaf}
% --------------------------------------------------------------------
Note that index accesibility filtering only makes sense for the one-phase
algorithm. See also \ref{sec:iaf} for more details about what this is.
\begin{code}

instanceIafAbleSimpleItemwhereiafAcc=siAccesibleiafInacc=siInaccessibleiafSetAccai=i{siAccesible=a}iafSetInaccai=i{siInaccessible=a}iafNewAcci=concatMapfromUniConst$concat[getIdxup|(TagSite_up__)<-siSubstnodesi]dpIafFailure::SimpleDispatchFilterdpIafFailureitem|auxitem=return$JustitemdpIafFailureitemRaw=dos<-getletbmap=semBitMapsitem=recalculateAccesibilityitemRawbadSem=iafBadSem(theIafMaps)bmap(tsems)siSemanticsiteminAcc=iafInaccitemifbadSem==0then-- can't dispatch, but that's good!-- (note that we return the item with its iaf criteria updated)return$JustitemelsedpToTrash(ts_iafFailureinAcc$bitVectorToSembmapbadSem)item

\end{code}
% --------------------------------------------------------------------
\section{Unpacking the results}
% --------------------------------------------------------------------
Unpacking the results consists of converting each result into a sentence
automaton (to take care of atomic disjunction) and reading the paths of
each automaton.
\begin{code}

\end{code}
\subsection{Sentence automata}
\fnlabel{listToSentenceAut} converts a list of GNodes into a sentence
automaton. It's a actually pretty stupid conversion in fact. We pretty
much make a straight path through the automaton, with the only
cleverness being that we provide a different transition for each
atomic disjunction.
\begin{code}

listToSentenceAut::[B.UninflectedDisjunction]->B.SentenceAutlistToSentenceAutnodes=lettheStart=0theEnd=(lengthnodes)-1theStates=[theStart..theEnd]--emptyAut=NFA{startSt=theStart,isFinalSt=Nothing,finalStList=[theEnd],states=[theStates],transitions=Map.empty}-- create a transition for each lexeme in the node to the-- next state...helper::(Int,B.UninflectedDisjunction)->B.SentenceAut->B.SentenceAuthelper(current,(lemmas,features))aut=foldl'addTautlemmaswhereaddTat=addTransacurrent(Just(t,features))nextnext=current+1--infoldrhelperemptyAut(ziptheStatesnodes)

\end{code}
% --------------------------------------------------------------------
\section{Partial results}
% --------------------------------------------------------------------
The user may ask for partial results when realisation fails. We implement this
using a greedy, full-commitment algorithm. Find the discarded result that
matches the largest part of the semantics and output that fragment. If there
are parts of the input semantics not covered by that fragment, search for the
largest chunk that covers the missing semantics. Recurse until there are no
more eligible items.
\begin{code}