TIP: 0Title: Tcl Core Team Basic Rules
Version: $Revision: 2.6 $
Author: John Ousterhout <ouster@pacbell.net>
State: Final
Type: Process
Vote: Done
Created: 11-Dec-2000
Post-History:
~ Abstract
This TIP describes the mission, structure, and operating procedures
of the Tcl Core Team (TCT). When in doubt about how the TCT works,
consult this document as the final authority.
~ Introduction
The Tcl Core Team is a self-organizing group of Tcl experts who are
responsible for the evolution and management of the Tcl core. The
Tcl Core Team decides what goes into releases; it implements, tests,
and documents new features and bug fixes; it manages the release
process; and it also manages the Tcl Developer Exchange Web site.
~ Scope: the Tcl core
The phrase "Tcl core" refers to the Tcl interpreter and the Tk
toolkit (the packages previously released by Sun, Scriptics, and
Ajuba). We also include the Tcl Developer Exchange Web site and
the Tcl bug database in the Tcl core. The Tcl Core Team may also
choose to take on additional responsibilities such as the creation
of more comprehensive "batteries included" releases. We expect
other Tcl development teams to form independently from the Tcl
Core Team to manage additional projects, such as popular extensions.
The Tcl Core Team may eventually spin off some of its activities
into separate teams.
~ Team membership
The Tcl Core Team is a small group of people who are making major
contributions to the development of the Tcl core and who are highly
respected and trusted by the Tcl community. Team members are expected
to invest significant amounts of their time to improve the Tcl core.
The original group of Team members was elected by the Tcl community,
................................................................................
participating in development projects under the auspices of an
existing team member.
Inactive or disruptive members of the team can be removed by a vote
of other Team members: a 2/3 majority of those voting is required to
remove a Team member.
~ Communication
The primary mechanism for communicating with the Tcl Core Team is the
mail alias tcl-core@lists.sourceforge.net. This is a public mailing
list; anyone interested in following the discussions of the TCT is
welcome to join the mailing list. Email sent to this alias is
archived, so you can review previous discussions at SourceForge.
~ Basic organizational structure
The team structure is simple and flat. All members have equal
standing: there is no Chairman. The Tcl Core Team makes its own
rules and chooses its own members as described in this document.
Anyone on the Tcl Core Team can propose a change in the rules;
after discussion, the change is voted on by the Team and must
receive 2/3 of the votes cast. The person proposing a rules change
is responsible for making sure that the change is properly
implemented after it has been approved (e.g. by modifying this
TIP, creating additional tools, etc.).
~ 2/3 vote
Wherever a 2/3 vote is called for in this document, it means that a
proposal must receive ''at least two-thirds of the votes cast'', not
votes from at least two-thirds of all TCT members.
~ Projects and maintainers
Tcl improvements are organized around two key ideas: ''projects''
and ''maintainers''. Most of the activities of the Tcl Core
Team consist of projects. A project can consist of a bug
fix, a new feature in the Tcl core, a new facility in the Tcl
Developer Exchange, or anything else except a change to this
TIP. We divide projects into two general categories: bug
fixes and feature changes. In general, if a project requires
manual entries to be updated then it is a feature change; when
in doubt, a project is a feature change . Bug fixes use a
................................................................................
more streamlined process for implementation, whereas feature
changes require discussion and approval in advance.
A maintainer is someone who has taken primary responsibility for
a portion of the Tcl sources. Many maintainers will be members of
the Tcl Core Team, but the Team may also select maintainers from
outside the Tcl Core Team. We hope to find enough maintainers to
cover all of the Tcl sources, but we will appoint a ''default
maintainer'' to handle the parts of Tcl for which no other maintainer
has volunteered. We'll also try to have backup maintainers who
can step in when the primary maintainers are on vacation or
otherwise unavailable.
A maintainer accepts several responsibilities, including the
following:
................................................................................
* Arranging for bugs to be fixed, either by doing it
himself/herself or finding someone else to do it.
* Coordinating and reviewing all modifications to his/her area.
* Providing assistance to other people working in his/her area.
~ Project life-cycle: approval, implementation, integration; TYANNOTT
The project for a feature change goes through three stages: approval,
implementation, and integration.
A project starts when a member of the Tcl Core Team proposes it to
the Team. Proposals are submitted by emailing TIPs (Tcl Improvement
Proposals) to the Tcl Core Team. The format of TIPs is described
in a separate TIP. Whoever proposes a project is responsible for
making sure it is properly implemented. A proposal without a
committed implementor cannot be approved.
Project approval is done through a process called ''TYANNOTT'':
Two Yesses And No No's Or Two Thirds. In order for a project to be
approved it must have support from at least one other member of the
Tcl Core Team besides the proposer. Once a project has been proposed
and discussed, if there are no objections and there is a vote of
confidence from a second team member ("Two Yesses And No No's"),
then the project is approved. If objections remain after the
discussion, then the proposer must summarize the objections and
call for a vote of the TCT; a 2/3 vote is required for approval.
The idea here is that most projects will be no-brainers and we want
a simple decision process that doesn't get in the way of progress.
On the other hand, the Tcl Core Team can only work effectively if
it is highly collegial; if the Team can't reach pretty clear
agreement on a project (i.e more than 1/3 of the TCT objects to it)
then the project needs to be rethought.
The second phase of a project is implementation. The proposer is
responsible for the implementation, either doing it himself/herself
or arranging for someone else to do it. The implementation is done
in a private work area and may not be integrated with the official
sources until the third phase, below.
The third phase of a project is integrating the results back into
the official Tcl repository. This is where maintainers come in.
First, before any change can be applied to the official Tcl sources,
the implementor must post it as a patch to the SourceForge patch
manager. This rule applies regardless of the type of change (anything
from a 1-line bug fix to a major new feature) and regardless of who
is proposing the change. We use the SourceForge patch manager to
record all changes and also to facilitate discussion about the
changes before they are applied.
When a patch arrives in the SourceForge patch manager, the
appropriate maintainer reviews it and works with the proposer
to revise it as necessary. Other people can also review the
patch, since it is public. If changes are needed, a revised
patch is logged in the patch manager (the final version of
the patch must always appear in the SourceForge patch manager).
Once the maintainer is satisfied with the patch, it can be applied
to the Tcl sources. If the patch implementor has write access
to the sources that he or she can apply the patch once the
maintainer has approved it. If the patch implementor doesn't
have write access to the sources than the maintainer applies
the patch.
................................................................................
If the implementor of a patch is the maintainer, then he/she can
apply the patch to the Tcl sources immediately after logging it in
the SourceForge patch manager, without waiting for additional
approval. However, if someone objects to the patch then the
maintainer must be prepared to revise it after the fact.
~ Fast path for bug fixes
For a bug fix, no initial proposal or approval is required. The
only approval needed is for the maintainer to review the patch
before it is applied to the sources. For example, we invite everyone
in the Tcl community to fix bugs and submit patches to the SourceForge
patch manager.
~ Implementors outside the Tcl Core Team
We encourage people outside the Tcl Core Team to get involved with
Tcl development. For example, anyone can submit patches for bug
fixes. It's also fine for someone outside the Tcl core team to
propose a feature change and then implement it, but there must be
a sponsor on the Tcl Core Team who will take personal responsibility
for it. Typically the sponsor will be the maintainer for the area
of the change. It is the sponsor's responsibility to provide whatever
level of supervision is appropriate to ensure that the project is
executed well. If the implementor for a project is not a TCT member
then they cannot vote for approval: TYANNOTT requires the sponsor
plus one other Team member.
~ Raising concerns
If you have concerns about a project, the best time to raise them is
during the initial discussion. Once a project has been approved, the
best approach is to raise the issue directly with the implementor;
most issues should get resolved quickly this way. If you can't find
the implementor or can't reach agreement, and if the implementor is
not a member of the Tcl Core Team, the next person to talk to is the
Tcl Core Team member in charge of the project. If you still can't get
satisfaction, then raise the issue with the entire Tcl Core Team by
leading a discussion. Once all the issues are out, you can either
withdraw your objection or summarize the issues (on both sides!) and
call for a vote. If you aren't a member of the Tcl Core Team
you will need to convince a Team member to manage the discussion
and vote.
Even if a project has received initial approval, a Team member can
object to the project later (e.g. if they believe it hasn't been
implemented properly). If the objection isn't resolved there will
be an additional vote of the Team, and the project cannot be applied
to the official sources unless it receives a 2/3 majority of
the votes cast. At the same time, Team members are expected to
raise their objections as early as possible; it would be somewhat
anti-social to raise a basic design objection late in the
implementation of a project when it could have been raised during
the initial approval.
~ Disagreements over patches
Normally, patches are not reviewed by the entire TCT; once the
relevant maintainer has reviewed and approved them then they can
be integrated. However, everyone is invited to review as many
patches as they wish. If someone on the TCT objects to a patch
and can't resolve the objection with the implementor and/or
maintainer, then it gets discussed by the entire Tcl Core Team with
................................................................................
Or, if someone on the Tcl Core Team objects to a patch applied by
a maintainer, they too can start a discussion in the whole team.
The goal of the maintainer mechanism is to simplify and speed up
improvements in the common case where everyone is in agreement,
while still allowing the entire Tcl Core Team to offer input and
resolve disagreements.
~ Changes that span areas
If a change involves several different areas of the Tcl sources,
with different maintainers, then one of the maintainers acts as
coordinator (presumably the one whose area has the most changes).
It is their responsibility to consult with other maintainers whose
areas are affected, so that all relevant maintainers are happy
before the patch is applied to the sources.
~ Write access to the Tcl sources and the Web site
Everyone in the Tcl Core Team has write access to all the sources
and the Web site, but they may only make changes consistent with
approved projects. The Tcl Core Team can also give access to other
people who are working on projects. For example, as part of a project
proposal a Tcl Core Team member can propose that the work will be
done by someone outside the team, and that that person should have
................................................................................
write access for putting back changes. Giving out write access is
part of a project decision, with the associated rules for approval.
However, if someone outside the Tcl Core Team has write access, it
must be under the auspices of a Tcl Core Team member; the Tcl
Core Team member is personally responsible for making sure the
project is completed satisfactorily and/or cleaning up any messes.
~ Deadlock resolution
If something should go wrong with the TCT organization and the
Tcl Core Team deadlocks to a point where it can't make meaningful
progress, then John Ousterhout will step in as benevolent dictator
and make enough unilateral decisions to break the deadlock.
~ Copyright
This document has been placed in the public domain.

# TIP 0: Tcl Core Team Basic Rules
Author: John Ousterhout <ouster@pacbell.net>
State: Final
Type: Process
Vote: Done
Created: 11-Dec-2000
Post-History:
-----# Abstract
This TIP describes the mission, structure, and operating procedures
of the Tcl Core Team \(TCT\). When in doubt about how the TCT works,
consult this document as the final authority.
# Introduction
The Tcl Core Team is a self-organizing group of Tcl experts who are
responsible for the evolution and management of the Tcl core. The
Tcl Core Team decides what goes into releases; it implements, tests,
and documents new features and bug fixes; it manages the release
process; and it also manages the Tcl Developer Exchange Web site.
# Scope: the Tcl core
The phrase "Tcl core" refers to the Tcl interpreter and the Tk
toolkit \(the packages previously released by Sun, Scriptics, and
Ajuba\). We also include the Tcl Developer Exchange Web site and
the Tcl bug database in the Tcl core. The Tcl Core Team may also
choose to take on additional responsibilities such as the creation
of more comprehensive "batteries included" releases. We expect
other Tcl development teams to form independently from the Tcl
Core Team to manage additional projects, such as popular extensions.
The Tcl Core Team may eventually spin off some of its activities
into separate teams.
# Team membership
The Tcl Core Team is a small group of people who are making major
contributions to the development of the Tcl core and who are highly
respected and trusted by the Tcl community. Team members are expected
to invest significant amounts of their time to improve the Tcl core.
The original group of Team members was elected by the Tcl community,
................................................................................
participating in development projects under the auspices of an
existing team member.
Inactive or disruptive members of the team can be removed by a vote
of other Team members: a 2/3 majority of those voting is required to
remove a Team member.
# Communication
The primary mechanism for communicating with the Tcl Core Team is the
mail alias tcl-core@lists.sourceforge.net. This is a public mailing
list; anyone interested in following the discussions of the TCT is
welcome to join the mailing list. Email sent to this alias is
archived, so you can review previous discussions at SourceForge.
# Basic organizational structure
The team structure is simple and flat. All members have equal
standing: there is no Chairman. The Tcl Core Team makes its own
rules and chooses its own members as described in this document.
Anyone on the Tcl Core Team can propose a change in the rules;
after discussion, the change is voted on by the Team and must
receive 2/3 of the votes cast. The person proposing a rules change
is responsible for making sure that the change is properly
implemented after it has been approved \(e.g. by modifying this
TIP, creating additional tools, etc.\).
# 2/3 vote
Wherever a 2/3 vote is called for in this document, it means that a
proposal must receive _at least two-thirds of the votes cast_, not
votes from at least two-thirds of all TCT members.
# Projects and maintainers
Tcl improvements are organized around two key ideas: _projects_
and _maintainers_. Most of the activities of the Tcl Core
Team consist of projects. A project can consist of a bug
fix, a new feature in the Tcl core, a new facility in the Tcl
Developer Exchange, or anything else except a change to this
TIP. We divide projects into two general categories: bug
fixes and feature changes. In general, if a project requires
manual entries to be updated then it is a feature change; when
in doubt, a project is a feature change . Bug fixes use a
................................................................................
more streamlined process for implementation, whereas feature
changes require discussion and approval in advance.
A maintainer is someone who has taken primary responsibility for
a portion of the Tcl sources. Many maintainers will be members of
the Tcl Core Team, but the Team may also select maintainers from
outside the Tcl Core Team. We hope to find enough maintainers to
cover all of the Tcl sources, but we will appoint a _default
maintainer_ to handle the parts of Tcl for which no other maintainer
has volunteered. We'll also try to have backup maintainers who
can step in when the primary maintainers are on vacation or
otherwise unavailable.
A maintainer accepts several responsibilities, including the
following:
................................................................................
* Arranging for bugs to be fixed, either by doing it
himself/herself or finding someone else to do it.
* Coordinating and reviewing all modifications to his/her area.
* Providing assistance to other people working in his/her area.
# Project life-cycle: approval, implementation, integration; TYANNOTT
The project for a feature change goes through three stages: approval,
implementation, and integration.
A project starts when a member of the Tcl Core Team proposes it to
the Team. Proposals are submitted by emailing TIPs \(Tcl Improvement
Proposals\) to the Tcl Core Team. The format of TIPs is described
in a separate TIP. Whoever proposes a project is responsible for
making sure it is properly implemented. A proposal without a
committed implementor cannot be approved.
Project approval is done through a process called _TYANNOTT_:
Two Yesses And No No's Or Two Thirds. In order for a project to be
approved it must have support from at least one other member of the
Tcl Core Team besides the proposer. Once a project has been proposed
and discussed, if there are no objections and there is a vote of
confidence from a second team member \("Two Yesses And No No's"\),
then the project is approved. If objections remain after the
discussion, then the proposer must summarize the objections and
call for a vote of the TCT; a 2/3 vote is required for approval.
The idea here is that most projects will be no-brainers and we want
a simple decision process that doesn't get in the way of progress.
On the other hand, the Tcl Core Team can only work effectively if
it is highly collegial; if the Team can't reach pretty clear
agreement on a project \(i.e more than 1/3 of the TCT objects to it\)
then the project needs to be rethought.
The second phase of a project is implementation. The proposer is
responsible for the implementation, either doing it himself/herself
or arranging for someone else to do it. The implementation is done
in a private work area and may not be integrated with the official
sources until the third phase, below.
The third phase of a project is integrating the results back into
the official Tcl repository. This is where maintainers come in.
First, before any change can be applied to the official Tcl sources,
the implementor must post it as a patch to the SourceForge patch
manager. This rule applies regardless of the type of change \(anything
from a 1-line bug fix to a major new feature\) and regardless of who
is proposing the change. We use the SourceForge patch manager to
record all changes and also to facilitate discussion about the
changes before they are applied.
When a patch arrives in the SourceForge patch manager, the
appropriate maintainer reviews it and works with the proposer
to revise it as necessary. Other people can also review the
patch, since it is public. If changes are needed, a revised
patch is logged in the patch manager \(the final version of
the patch must always appear in the SourceForge patch manager\).
Once the maintainer is satisfied with the patch, it can be applied
to the Tcl sources. If the patch implementor has write access
to the sources that he or she can apply the patch once the
maintainer has approved it. If the patch implementor doesn't
have write access to the sources than the maintainer applies
the patch.
................................................................................
If the implementor of a patch is the maintainer, then he/she can
apply the patch to the Tcl sources immediately after logging it in
the SourceForge patch manager, without waiting for additional
approval. However, if someone objects to the patch then the
maintainer must be prepared to revise it after the fact.
# Fast path for bug fixes
For a bug fix, no initial proposal or approval is required. The
only approval needed is for the maintainer to review the patch
before it is applied to the sources. For example, we invite everyone
in the Tcl community to fix bugs and submit patches to the SourceForge
patch manager.
# Implementors outside the Tcl Core Team
We encourage people outside the Tcl Core Team to get involved with
Tcl development. For example, anyone can submit patches for bug
fixes. It's also fine for someone outside the Tcl core team to
propose a feature change and then implement it, but there must be
a sponsor on the Tcl Core Team who will take personal responsibility
for it. Typically the sponsor will be the maintainer for the area
of the change. It is the sponsor's responsibility to provide whatever
level of supervision is appropriate to ensure that the project is
executed well. If the implementor for a project is not a TCT member
then they cannot vote for approval: TYANNOTT requires the sponsor
plus one other Team member.
# Raising concerns
If you have concerns about a project, the best time to raise them is
during the initial discussion. Once a project has been approved, the
best approach is to raise the issue directly with the implementor;
most issues should get resolved quickly this way. If you can't find
the implementor or can't reach agreement, and if the implementor is
not a member of the Tcl Core Team, the next person to talk to is the
Tcl Core Team member in charge of the project. If you still can't get
satisfaction, then raise the issue with the entire Tcl Core Team by
leading a discussion. Once all the issues are out, you can either
withdraw your objection or summarize the issues \(on both sides!\) and
call for a vote. If you aren't a member of the Tcl Core Team
you will need to convince a Team member to manage the discussion
and vote.
Even if a project has received initial approval, a Team member can
object to the project later \(e.g. if they believe it hasn't been
implemented properly\). If the objection isn't resolved there will
be an additional vote of the Team, and the project cannot be applied
to the official sources unless it receives a 2/3 majority of
the votes cast. At the same time, Team members are expected to
raise their objections as early as possible; it would be somewhat
anti-social to raise a basic design objection late in the
implementation of a project when it could have been raised during
the initial approval.
# Disagreements over patches
Normally, patches are not reviewed by the entire TCT; once the
relevant maintainer has reviewed and approved them then they can
be integrated. However, everyone is invited to review as many
patches as they wish. If someone on the TCT objects to a patch
and can't resolve the objection with the implementor and/or
maintainer, then it gets discussed by the entire Tcl Core Team with
................................................................................
Or, if someone on the Tcl Core Team objects to a patch applied by
a maintainer, they too can start a discussion in the whole team.
The goal of the maintainer mechanism is to simplify and speed up
improvements in the common case where everyone is in agreement,
while still allowing the entire Tcl Core Team to offer input and
resolve disagreements.
# Changes that span areas
If a change involves several different areas of the Tcl sources,
with different maintainers, then one of the maintainers acts as
coordinator \(presumably the one whose area has the most changes\).
It is their responsibility to consult with other maintainers whose
areas are affected, so that all relevant maintainers are happy
before the patch is applied to the sources.
# Write access to the Tcl sources and the Web site
Everyone in the Tcl Core Team has write access to all the sources
and the Web site, but they may only make changes consistent with
approved projects. The Tcl Core Team can also give access to other
people who are working on projects. For example, as part of a project
proposal a Tcl Core Team member can propose that the work will be
done by someone outside the team, and that that person should have
................................................................................
write access for putting back changes. Giving out write access is
part of a project decision, with the associated rules for approval.
However, if someone outside the Tcl Core Team has write access, it
must be under the auspices of a Tcl Core Team member; the Tcl
Core Team member is personally responsible for making sure the
project is completed satisfactorily and/or cleaning up any messes.
# Deadlock resolution
If something should go wrong with the TCT organization and the
Tcl Core Team deadlocks to a point where it can't make meaningful
progress, then John Ousterhout will step in as benevolent dictator
and make enough unilateral decisions to break the deadlock.
# Copyright
This document has been placed in the public domain.

TIP: 1Title: TIP IndexVersion: $Revision: 1.6 $
Author: TIP Editor <donal.fellows@cs.man.ac.uk>
State: Active
Type: Informational
Vote: No voting
Created: 14-Sep-2000
Post-History:
~ Abstract
This TIP contains the index of all TIPs published over the lifetime of
the TCT. It will be continually and automatically updated.
~ Index
#index:
White backgrounds indicate that the TIP is still a draft, yellow
backgrounds highlight TIPs being voted on, and where a TIP has been
rejected, withdrawn or obsoleted its index entry has a dark grey
background. Blue backgrounds indicate a TIP has been accepted,
but still needs an implementation approved by maintainers. Green
backgrounds indicate that the TIP is deferred, waiting for someone
to work on it.
~ Explanations and How To Submit New TIPs
See [2] for a description of the editorial process a TIP has to go
through and [3] for a description of their structure and the commands
used to write them. You submit a TIP to this archive by emailing it
(preferably in source form) to the TIP editor <donal.fellows@man.ac.uk>
who will check it for following of the guidelines, style and general
relevance to Tcl/Tk before checking it into the CVS archive and notifying
the author, the rest of the Tcl Core Team, and the relevant newsgroups.
~ Copyright
This document has been placed in the public domain.

# TIP 1: TIP IndexAuthor: TIP Editor <donal.fellows@cs.man.ac.uk>
State: Active
Type: Informational
Vote: No voting
Created: 14-Sep-2000
Post-History:
-----# Abstract
This TIP contains the index of all TIPs published over the lifetime of
the TCT. It will be continually and automatically updated.
# Index
\#index:
White backgrounds indicate that the TIP is still a draft, yellow
backgrounds highlight TIPs being voted on, and where a TIP has been
rejected, withdrawn or obsoleted its index entry has a dark grey
background. Blue backgrounds indicate a TIP has been accepted,
but still needs an implementation approved by maintainers. Green
backgrounds indicate that the TIP is deferred, waiting for someone
to work on it.
# Explanations and How To Submit New TIPs
See [[2]](2.md) for a description of the editorial process a TIP has to go
through and [[3]](3.md) for a description of their structure and the commands
used to write them. You submit a TIP to this archive by emailing it
\(preferably in source form\) to the TIP editor <donal.fellows@man.ac.uk>
who will check it for following of the guidelines, style and general
relevance to Tcl/Tk before checking it into the CVS archive and notifying
the author, the rest of the Tcl Core Team, and the relevant newsgroups.
# Copyright
This document has been placed in the public domain.

TIP: 10Title: Tcl I/O Enhancement: Thread-Aware Channels
Version: $Revision: 1.6 $
Author: Andreas Kupries <a.kupries@westend.com>
State: Final
Type: Project
Vote: Done
Created: 08-Nov-2000
Post-History:
Tcl-Version: 8.4
~ Abstract
This TIP describes how to change the generic I/O layer in the Tcl core
to make channels aware of the thread they are managed by.
~ Rationale
To explain the motives behind this TIP first a short look at the
history of channels and threading.
In ancient times the Tcl core was not thread safe and did not employ
threads. All channels belonged to a single interpreter. Later on
interpreter hierarchies were introduced and the ability to move or
share a channel between the interpreters in a hierarchy. When the Tcl
core was made thread safe a short time after the ability to move
channels between threads was added (Helper APIs in the core, main
functionality in the Thread extension). The goal behind these
modifications was to enable the creation of stream-like communication
paths between threads to complement the message based facilities
(thread send). The modifications were only a partial success because
an in-depth analysis of the relevant data structures showed that the
sharing of a channel between threads is not possible with the current
design, only moving. This was implemented to allow at least the
dispatcher- / worker-thread pattern for structuring a threaded
application.
In further pursuit of the original goal the currently chosen approach
is to define a channeltype where two channels are connected internally
through in-memory fifo buffers where access to these shared structures
is protected by mutexes.
During the implementation of fileevents for this channeltype it was
discovered that an efficient implementation of this part is ''not''
possible because of the inability to post file events to the
eventqueue of the thread the other channel of the pair resides in. An
API to post such events is available (''Tcl_ThreadQueueEvent''), but
not the information which thread actually manages the other
channel. Because of this the current implementation of the channeltype
uses polling based upon timer events posted by each side/thread to
itself to manage file events in a rather inefficient way.
~ Reference implementation
This TIP now proposes to change the internals of the generic I/O
layers in the core so that
1. Channels know the thread they are managed by, and
1. are able to deliver this information to an extension querying
................................................................................
This then allows the two sides of the channeltype mentioned above to
post events to each other, facilitating an efficient implementation of
fileevents.
The changes necessary to accomplish this are:
1. Extend the structure ''ChannelState'' in tclIO.h with a new
field of type ''Tcl_ThreadId'' to hold the id of the thread
currently managing all channels with this state. Note: This
structure is shared by all channels in a stack of
transformations.
1. Modify the procedure ''Tcl_CreateChannel'' to store the Id of
the current thread in the ''ChannelState'' of the new channel.
This information can be obtained with ''Tcl_GetCurrentThread''.
It is ''not'' necessary to modify ''Tcl_StackChannel'' as the
thread information is already part of the state when it is
called, and won't be changed by the call.
1. If some sort of NIL/NULL value meaning "No thread" is available
for ''Tcl_ThreadId'', then we should modify ''Tcl_CutChannel''
to insert this value into the state of the channel it is called
with, as this channel will not be managed by any thread
afterward (the procedure removes the channel from the list of
all channels managed by the current thread).
1. Modify ''Tcl_SpliceChannel'' in the same manner as
''Tcl_CreateChannel'' as the channel will be managed by the
current thread afterward (The procedure adds the channel to the
list of all channels managed by the current thread).
1. Declare a new API function to retrieve the Id of the managing
thread from a channel. Add this declaration to generic/tcl.decls
and implement the function in the file generic/tclIO.c. I
propose ''Tcl_GetChannelThread'' as the name of this new API
function.
A patch implementing all of the changes described above and
additionally extending the documentation and the test-suite
is available here:
http://www.cs.man.ac.uk/fellowsd-bin/TIP/10.patch
~ Copyright
This document has been placed in the public domain.

# TIP 10: Tcl I/O Enhancement: Thread-Aware Channels
Author: Andreas Kupries <a.kupries@westend.com>
State: Final
Type: Project
Vote: Done
Created: 08-Nov-2000
Post-History:
Tcl-Version: 8.4
-----# Abstract
This TIP describes how to change the generic I/O layer in the Tcl core
to make channels aware of the thread they are managed by.
# Rationale
To explain the motives behind this TIP first a short look at the
history of channels and threading.
In ancient times the Tcl core was not thread safe and did not employ
threads. All channels belonged to a single interpreter. Later on
interpreter hierarchies were introduced and the ability to move or
share a channel between the interpreters in a hierarchy. When the Tcl
core was made thread safe a short time after the ability to move
channels between threads was added \(Helper APIs in the core, main
functionality in the Thread extension\). The goal behind these
modifications was to enable the creation of stream-like communication
paths between threads to complement the message based facilities
\(thread send\). The modifications were only a partial success because
an in-depth analysis of the relevant data structures showed that the
sharing of a channel between threads is not possible with the current
design, only moving. This was implemented to allow at least the
dispatcher- / worker-thread pattern for structuring a threaded
application.
In further pursuit of the original goal the currently chosen approach
is to define a channeltype where two channels are connected internally
through in-memory fifo buffers where access to these shared structures
is protected by mutexes.
During the implementation of fileevents for this channeltype it was
discovered that an efficient implementation of this part is _not_
possible because of the inability to post file events to the
eventqueue of the thread the other channel of the pair resides in. An
API to post such events is available \(_Tcl\_ThreadQueueEvent_\), but
not the information which thread actually manages the other
channel. Because of this the current implementation of the channeltype
uses polling based upon timer events posted by each side/thread to
itself to manage file events in a rather inefficient way.
# Reference implementation
This TIP now proposes to change the internals of the generic I/O
layers in the core so that
1. Channels know the thread they are managed by, and
1. are able to deliver this information to an extension querying
................................................................................
This then allows the two sides of the channeltype mentioned above to
post events to each other, facilitating an efficient implementation of
fileevents.
The changes necessary to accomplish this are:
1. Extend the structure _ChannelState_ in tclIO.h with a new
field of type _Tcl\_ThreadId_ to hold the id of the thread
currently managing all channels with this state. Note: This
structure is shared by all channels in a stack of
transformations.
1. Modify the procedure _Tcl\_CreateChannel_ to store the Id of
the current thread in the _ChannelState_ of the new channel.
This information can be obtained with _Tcl\_GetCurrentThread_.
It is _not_ necessary to modify _Tcl\_StackChannel_ as the
thread information is already part of the state when it is
called, and won't be changed by the call.
1. If some sort of NIL/NULL value meaning "No thread" is available
for _Tcl\_ThreadId_, then we should modify _Tcl\_CutChannel_
to insert this value into the state of the channel it is called
with, as this channel will not be managed by any thread
afterward \(the procedure removes the channel from the list of
all channels managed by the current thread\).
1. Modify _Tcl\_SpliceChannel_ in the same manner as
_Tcl\_CreateChannel_ as the channel will be managed by the
current thread afterward \(The procedure adds the channel to the
list of all channels managed by the current thread\).
1. Declare a new API function to retrieve the Id of the managing
thread from a channel. Add this declaration to generic/tcl.decls
and implement the function in the file generic/tclIO.c. I
propose _Tcl\_GetChannelThread_ as the name of this new API
function.
A patch implementing all of the changes described above and
additionally extending the documentation and the test-suite
is available here:
<http://www.cs.man.ac.uk/fellowsd-bin/TIP/10.patch># Copyright
This document has been placed in the public domain.

TIP: 100Title: Add Support for Unloading Dynamic Libraries Loaded with [load]
Version: $Revision: 1.10 $
Author: George Petasis <petasis@iit.demokritos.gr>
State: Final
Type: Project
Vote: Done
Created: 11-Jun-2002
Post-History:
Discussions-To: news:comp.lang.tcl
Keywords: load,unload,dynamic library
Tcl-Version: 8.5
~ Abstract
Tcl already provides facilities for loading dynamic libraries, through
the ''load'' command. However no facilities are currently offered in
order to ''unload'' dynamic libraries already loaded with the ''load''
command. This TIP tries to add support for unloading libraries, by
introducing a new Tcl command (''unload'') and the guidelines that
dynamic libraries must follow, in order to be unloadable. Note that
the ''unload'' command will operate only on libraries that are
designed to be unloadable by their developers. This way backward
compatibility with older extensions is maintained, as unload will
never try to unload libraries unaware of this new functionality.
~ Rationale
Tcl is an ideal language for component-based applications. Usually
these applications offer a framework in which components developed by
the users of the application can be embedded, in order to extend the
functionality of the framework. Usually, these extensions are
implemented as C/C++ dynamic libraries that are loaded through the
''load'' Tcl command.
However the development of such components can be a time-consuming
process, as developers have to restart the framework application in
order to be able to reload the library into it and test its altered
functionality. And this can be quite annoying (depending on the
application of course), as usually processing within the application
is required in order to bring it into a proper state before testing
the library.
The development cycle can be significantly shortened if Tcl provides a
mechanism for unloading a dynamic library. A new version of the
library can be created, as its object file can now be written, and the
updated library can be re-loaded.
However, this is not the only application of unload.
Services running for long periods of time and want to unload
no longer needed functionality, replacing parts of an applications
(i.e. from an automatic update procedure) or functionality
temporarily needed (i.e. a web browser that loads a plugin
to display a file of a particular file type) are some
additional fields of application.
~ Introduction
Unload functionality has been left out of the Tcl core, mainly because
library unloading was poorly implemented in many operating systems.
But almost all operating systems have been improved in the meantime,
and as a result most modern operating systems now support library
unloading.
The main idea of this TIP is to enable dynamic library unloading at
the Tcl level, in the same sense ''load'' provides dynamic library
loading. However, library unloading will be provided only when the
underlying operating system support this feature (as is also the case
for ''load'') and only when the library to be unloaded provides a set
of functions that can "undo" the changes the library has made to the
interpreter. In all other cases, unloading a library will result in
an error.
This TIP proposes the insertion of a new Tcl command named ''unload''
and two functions ''pkg_Unload'' and ''pkg_SafeUnload'', modelled
after ''pkg_Init'' and ''pkg_SafeInit'', that libraries which can be
unloaded should implement.
~ Unloadable Libraries
A main concern is related to when a shared library can be "unloadable".
An unloadable library is a library characterised as such by its
developer. The developer of the library must export a function
from the library, similar to the library's initialisation
function. The unload command will never try to unload a library
that does not provide such a function. This makes old libraries
(before the introduction of the unload functionality) safe.
There is a category of libraries that
can never be unloaded (i.e. libraries that register new
tcl object types). However, the choice is upon the developer:
the developer knows if the library can be unloadable. The simpler
case, libraries that only register new commands are the most
probable libraries to be unloadable.
Libraries that export functions through a stub mechanism cannot be
unloaded, as the were meant for having dependencies with
other libraries that use the exported API. There is no way for
the provider library to know wether it is used or not.
~ Specification
Actually, all the facilities for unloading dynamic libraries already
exist in the Tcl core and simply they are not yet exposed at the Tcl
level. As a result, the implementation of the unload command should
be fairly easy.
''load'' as it is currently implemented loads a dynamic library only
the first time this library is loaded. It keeps an internal cache of
all loaded libraries and if an already loaded library is requested
again, only its initialisation function is called. This cache should
be extended to keep some additional information:
1. Two reference counts, counting how many times a specific
library has been loaded. This reference count should be
increased by each ''load'' and decreased for each ''unload''.
When it reaches 0, the library can be unloaded. Safe
interpreters and normal interpreters should have different
reference counts. Both should be 0 in order for a library to
be unloaded.
2. The addresses of the ''pkg_Unload'' and ''pkg_SafeUnload''
functions, if these are implemented by the library. If both of
these functions are missing, the library will never be
unloaded. If only ''pkg_Unload'' is implemented, the library
can be unloaded if it never has been loaded in a safe
interpreter. Finally, if ''pkg_SafeUnload'' is implemented,
the library can be unloaded if it has never been loaded in a
normal interpreter.
The ''unload'' command will always return an error, if the operating
system does not support library unloading. In case the operating
system supports library unloading:
1. ''unload'' will examine the cache of ''load'' to locate the
entry for the library to be unloaded. It is an error to unload
a library that has not been loaded with ''load''.
2. If the entry in the cache is found, ''unload'' checks whether
the corresponding for the interpreter type unload function
pointer is NULL or not. If it is NULL, the library cannot be
unloaded under this interpreter and again an error is returned.
3. If the unload function pointer is not NULL, it is executed. If
an error is returned by this function, ''unload'' also returns
an error.
4. If the unload function finishes without errors, the reference
count corresponding to the interpreter type is decreased. If
both reference counts reach 0, the library is unloaded.
~ Responsibilities of the Unload Functions
Its up to the developer of the library to decide if its library can be
unloaded or not. A library can be unloaded if the function
''pkg_Unload'' is implemented and exported as a symbol from the
library, and the library will never be loaded in a safe interpreter.
A library that can be also loaded in safe interpreters is unloadable
if the function ''pkg_SafeUnload'' is also available. These two
functions will accept two arguments, the interpreter under which the
library is unloaded and an integer, holding various flags. The flags argument
can be either ''TCL_UNLOAD_DETACH_FROM_INTERPRETER'' or
''TCL_UNLOAD_DETACH_FROM_PROCESS''. In case the library will remain attached to
the process after the unload procedure returns (i.e. because the library is
used by other interpreters), TCL_UNLOAD_DETACH_FROM_INTERPRETER will be defined.
However, if the library is used only by the target interpreter and the library
will be detached from the application as soon as the unload procedure returns,
the flags argument will be set to TCL_UNLOAD_DETACH_FROM_PROCESS.
The main responsibility of these functions is to remove from the
interpreter they are unloaded under any reference to a function
residing inside the library. For example, such a function must:
1. Unregister any commands that have been registered by the
''Init()'' function to the ''interpreter'' given by the first
argument. In order to do this, the library should keep
internally the tokens returned by each ''Tcl_Create*Command'',
as command may have been renamed.
2. Unregister any other commands that may have been registered to
the interpreter during the use of the library (usually used to
represent special special data structures).
If the flag ''TCL_UNLOAD_DETACH_FROM_PROCESS'' is defined, the
developer must do additional task, that are not normally required when
the library gets unloaded from an interpreter:
3. Free any memory occupied by the internal structures of
the library.
4. In general, try to remove any references Tcl may have
to functions provided by the library.
If the developer cannot remove all reference to functions to the
library, its better to not provide at all these two functions, so as
unload to never attempt to unload the library.
~ Dependencies Among Libraries
It is possible that a library A has been loaded that exports some
symbols. Then a library B is loaded, that has dependencies (i.e. uses
some exported symbols) on A. What if A gets unloaded?
Actually, most modern operating systems seem to provide a solution to
this problem, as reference counts are hold internally by the operating
system for each library. Newer Windows, Solaris and Linux seem to
provide similar solutions and in reality they don't unload the library
if such symbols remain, even if the unload system call has been made
for the library. Both libraries A and B have to be unloaded in order
for A to be really removed from the address space.
~ Reference Implementation
A reference implementation can be found at:
http://sf.net/tracker/?func=detail&aid=823486&group_id=10894&atid=310894~ Copyright
This document has been placed in the public domain.
~ Appendix: The unload man page.
NAME
unload - Unload machine code.
SYNOPSIS
unload ?switches? fileName
unload ?switches? fileName packageName
unload ?switches? fileName packageName interp
................................................................................
This command tries to unload shared libraries previously
loaded with load from the application's address space.
fileName is the name of the file containing the library
file to be unload; it must be the same as the filename
provided to load for loading the library. packageName is
the name of the package, and is used to compute the name
of the unload procedure. interp is the path name of the
interpreter from which to unload the package (see the
interp manual entry for details); if interp is omitted,
it defaults to the interpreter in which the unload
command was invoked.
If the initial arguments to unload start with - then they
are treated as switches. The following switches are
currently supported:
................................................................................
Marks the end of switches. The argument following
this one will be treated as a fileName even if it
starts with a -.
When a file containing a shared library is loaded through
the load command, Tcl associates two reference counts to
the library file. The first counter shows how many times
the library has been loaded into normal (trusted)
interpreters while the second describes how many times
the library has been loaded into safe interpreters. As a
file containing a shared library can be loaded only once
by Tcl (with the first load call on the file), these
counters track how many interpreters use the library.
Each subsequent call to load after the first, simply
increaments the proper reference count.
unload works in the opposite direction. As a first step,
unload will check whether the library is unloadable: an
unloadable library exports a special unload procedure.
The name of the unload procedure is determined by
packageName and whether or not the target interpreter is
a safe one. For normal interpreters the name of the
initialization procedure will have the form pkg_Unload,
where pkg is the same as packageName except that the
first letter is converted to upper case and all other
letters are converted to lower case. For example, if
packageName is foo or FOo, the initialization procedure's
name will be Foo_Unload. If the target interpreter is a
safe interpreter, then the name of the initialization
procedure will be pkg_SafeUnload instead of pkg_Unload.
If unload determines that a library is not unloadable (or
unload functionality has been disabled during
compilation), an error will be returned. If the library
is unloadable, then unload will call the unload
procedure. If the unload procedure returns TCL_OK, unload
will proceed and decrease the proper reference count
(depending on the target interpreter type). When both
reference counts have reached 0, the library will be
detached from the process.
The unload procedure must match the following prototype:
typedef int Tcl_PackageUnloadProc(Tcl_Interp *interp, int
flags);
The interp argument identifies the interpreter from which
the library is to be unloaded. The unload procedure must
return TCL_OK or TCL_ERROR to indicate whether or not it
completed successfully; in the event of an error it
should set the interpreter's result to point to an error
message. In this case, the result of the unload command
will be the result returned by the unload procedure. The
flags argument can be either
TCL_UNLOAD_DETACH_FROM_INTERPRETER or
TCL_UNLOAD_DETACH_FROM_PROCESS. In case the library will
remain attached to the process after the unload procedure
returns (i.e. because the library is used by other
interpreters), TCL_UNLOAD_DETACH_FROM_INTERPRETER will be
defined. However, if the library is used only by the
target interpreter and the library will be detached from
the application as soon as the unload procedure returns,
the flags argument will be set to
TCL_UNLOAD_DETACH_FROM_PROCESS.
The unload command cannot unload libraries that are
statically linked with the application. If fileName is an
empty string, then packageName must be specified.
If packageName is omitted or specified as an empty
string, Tcl tries to guess the name of the package. This
may be done differently on different platforms. The
default guess, which is used on most UNIX platforms, is
to take the last element of fileName, strip off the first
three characters if they are lib, and use any following
alphabetic and underline characters as the module name.
For example, the command unload libxyz4.2.so uses the
module name xyz and the command unload bin/last.so {}
uses the module name last.
PORTABILITY ISSUES
Unix
Not all unix operating systems support library
unloading. Under such an operating system unload
returns an error (unless -nocomplain has been
specified).
Macintosh
<Somebody to comment on this?>
BUGS
If the same file is loaded by different fileNames, it
will be loaded into the process's address space multiple
times. The behavior of this varies from system to system
(some systems may detect the redundant loads, others may
not). In case a library has been silently detached by the
operating system (and as a result Tcl thinks the library
is still loaded), it may be dangerous to use unload on
such a library (as the library will be completely
detached from the application while some interpreters
will continue to use it).
SEE ALSO
info sharedlibextension, load, safe
KEYWORDS
binary code, unloading, safe interpreter, shared library

# TIP 100: Add Support for Unloading Dynamic Libraries Loaded with [load]
Author: George Petasis <petasis@iit.demokritos.gr>
State: Final
Type: Project
Vote: Done
Created: 11-Jun-2002
Post-History:
Discussions-To: news:comp.lang.tcl
Keywords: load,unload,dynamic library
Tcl-Version: 8.5
-----# Abstract
Tcl already provides facilities for loading dynamic libraries, through
the _load_ command. However no facilities are currently offered in
order to _unload_ dynamic libraries already loaded with the _load_
command. This TIP tries to add support for unloading libraries, by
introducing a new Tcl command \(_unload_\) and the guidelines that
dynamic libraries must follow, in order to be unloadable. Note that
the _unload_ command will operate only on libraries that are
designed to be unloadable by their developers. This way backward
compatibility with older extensions is maintained, as unload will
never try to unload libraries unaware of this new functionality.
# Rationale
Tcl is an ideal language for component-based applications. Usually
these applications offer a framework in which components developed by
the users of the application can be embedded, in order to extend the
functionality of the framework. Usually, these extensions are
implemented as C/C\+\+ dynamic libraries that are loaded through the
_load_ Tcl command.
However the development of such components can be a time-consuming
process, as developers have to restart the framework application in
order to be able to reload the library into it and test its altered
functionality. And this can be quite annoying \(depending on the
application of course\), as usually processing within the application
is required in order to bring it into a proper state before testing
the library.
The development cycle can be significantly shortened if Tcl provides a
mechanism for unloading a dynamic library. A new version of the
library can be created, as its object file can now be written, and the
updated library can be re-loaded.
However, this is not the only application of unload.
Services running for long periods of time and want to unload
no longer needed functionality, replacing parts of an applications
\(i.e. from an automatic update procedure\) or functionality
temporarily needed \(i.e. a web browser that loads a plugin
to display a file of a particular file type\) are some
additional fields of application.
# Introduction
Unload functionality has been left out of the Tcl core, mainly because
library unloading was poorly implemented in many operating systems.
But almost all operating systems have been improved in the meantime,
and as a result most modern operating systems now support library
unloading.
The main idea of this TIP is to enable dynamic library unloading at
the Tcl level, in the same sense _load_ provides dynamic library
loading. However, library unloading will be provided only when the
underlying operating system support this feature \(as is also the case
for _load_\) and only when the library to be unloaded provides a set
of functions that can "undo" the changes the library has made to the
interpreter. In all other cases, unloading a library will result in
an error.
This TIP proposes the insertion of a new Tcl command named _unload_
and two functions _pkg\_Unload_ and _pkg\_SafeUnload_, modelled
after _pkg\_Init_ and _pkg\_SafeInit_, that libraries which can be
unloaded should implement.
# Unloadable Libraries
A main concern is related to when a shared library can be "unloadable".
An unloadable library is a library characterised as such by its
developer. The developer of the library must export a function
from the library, similar to the library's initialisation
function. The unload command will never try to unload a library
that does not provide such a function. This makes old libraries
\(before the introduction of the unload functionality\) safe.
There is a category of libraries that
can never be unloaded \(i.e. libraries that register new
tcl object types\). However, the choice is upon the developer:
the developer knows if the library can be unloadable. The simpler
case, libraries that only register new commands are the most
probable libraries to be unloadable.
Libraries that export functions through a stub mechanism cannot be
unloaded, as the were meant for having dependencies with
other libraries that use the exported API. There is no way for
the provider library to know wether it is used or not.
# Specification
Actually, all the facilities for unloading dynamic libraries already
exist in the Tcl core and simply they are not yet exposed at the Tcl
level. As a result, the implementation of the unload command should
be fairly easy.
_load_ as it is currently implemented loads a dynamic library only
the first time this library is loaded. It keeps an internal cache of
all loaded libraries and if an already loaded library is requested
again, only its initialisation function is called. This cache should
be extended to keep some additional information:
1. Two reference counts, counting how many times a specific
library has been loaded. This reference count should be
increased by each _load_ and decreased for each _unload_.
When it reaches 0, the library can be unloaded. Safe
interpreters and normal interpreters should have different
reference counts. Both should be 0 in order for a library to
be unloaded.
2. The addresses of the _pkg\_Unload_ and _pkg\_SafeUnload_
functions, if these are implemented by the library. If both of
these functions are missing, the library will never be
unloaded. If only _pkg\_Unload_ is implemented, the library
can be unloaded if it never has been loaded in a safe
interpreter. Finally, if _pkg\_SafeUnload_ is implemented,
the library can be unloaded if it has never been loaded in a
normal interpreter.
The _unload_ command will always return an error, if the operating
system does not support library unloading. In case the operating
system supports library unloading:
1. _unload_ will examine the cache of _load_ to locate the
entry for the library to be unloaded. It is an error to unload
a library that has not been loaded with _load_.
2. If the entry in the cache is found, _unload_ checks whether
the corresponding for the interpreter type unload function
pointer is NULL or not. If it is NULL, the library cannot be
unloaded under this interpreter and again an error is returned.
3. If the unload function pointer is not NULL, it is executed. If
an error is returned by this function, _unload_ also returns
an error.
4. If the unload function finishes without errors, the reference
count corresponding to the interpreter type is decreased. If
both reference counts reach 0, the library is unloaded.
# Responsibilities of the Unload Functions
Its up to the developer of the library to decide if its library can be
unloaded or not. A library can be unloaded if the function
_pkg\_Unload_ is implemented and exported as a symbol from the
library, and the library will never be loaded in a safe interpreter.
A library that can be also loaded in safe interpreters is unloadable
if the function _pkg\_SafeUnload_ is also available. These two
functions will accept two arguments, the interpreter under which the
library is unloaded and an integer, holding various flags. The flags argument
can be either _TCL\_UNLOAD\_DETACH\_FROM\_INTERPRETER_ or
_TCL\_UNLOAD\_DETACH\_FROM\_PROCESS_. In case the library will remain attached to
the process after the unload procedure returns \(i.e. because the library is
used by other interpreters\), TCL\_UNLOAD\_DETACH\_FROM\_INTERPRETER will be defined.
However, if the library is used only by the target interpreter and the library
will be detached from the application as soon as the unload procedure returns,
the flags argument will be set to TCL\_UNLOAD\_DETACH\_FROM\_PROCESS.
The main responsibility of these functions is to remove from the
interpreter they are unloaded under any reference to a function
residing inside the library. For example, such a function must:
1. Unregister any commands that have been registered by the
_Init\(\)_ function to the _interpreter_ given by the first
argument. In order to do this, the library should keep
internally the tokens returned by each _Tcl\_Create\*Command_,
as command may have been renamed.
2. Unregister any other commands that may have been registered to
the interpreter during the use of the library \(usually used to
represent special special data structures\).
If the flag _TCL\_UNLOAD\_DETACH\_FROM\_PROCESS_ is defined, the
developer must do additional task, that are not normally required when
the library gets unloaded from an interpreter:
3. Free any memory occupied by the internal structures of
the library.
4. In general, try to remove any references Tcl may have
to functions provided by the library.
If the developer cannot remove all reference to functions to the
library, its better to not provide at all these two functions, so as
unload to never attempt to unload the library.
# Dependencies Among Libraries
It is possible that a library A has been loaded that exports some
symbols. Then a library B is loaded, that has dependencies \(i.e. uses
some exported symbols\) on A. What if A gets unloaded?
Actually, most modern operating systems seem to provide a solution to
this problem, as reference counts are hold internally by the operating
system for each library. Newer Windows, Solaris and Linux seem to
provide similar solutions and in reality they don't unload the library
if such symbols remain, even if the unload system call has been made
for the library. Both libraries A and B have to be unloaded in order
for A to be really removed from the address space.
# Reference Implementation
A reference implementation can be found at:
<http://sf.net/tracker/?func=detail&aid=823486&group\_id=10894&atid=310894># Copyright
This document has been placed in the public domain.
# Appendix: The unload man page.
NAME
unload - Unload machine code.
SYNOPSIS
unload ?switches? fileName
unload ?switches? fileName packageName
unload ?switches? fileName packageName interp
................................................................................
This command tries to unload shared libraries previously
loaded with load from the application's address space.
fileName is the name of the file containing the library
file to be unload; it must be the same as the filename
provided to load for loading the library. packageName is
the name of the package, and is used to compute the name
of the unload procedure. interp is the path name of the
interpreter from which to unload the package \(see the
interp manual entry for details\); if interp is omitted,
it defaults to the interpreter in which the unload
command was invoked.
If the initial arguments to unload start with - then they
are treated as switches. The following switches are
currently supported:
................................................................................
Marks the end of switches. The argument following
this one will be treated as a fileName even if it
starts with a -.
When a file containing a shared library is loaded through
the load command, Tcl associates two reference counts to
the library file. The first counter shows how many times
the library has been loaded into normal \(trusted\)
interpreters while the second describes how many times
the library has been loaded into safe interpreters. As a
file containing a shared library can be loaded only once
by Tcl \(with the first load call on the file\), these
counters track how many interpreters use the library.
Each subsequent call to load after the first, simply
increaments the proper reference count.
unload works in the opposite direction. As a first step,
unload will check whether the library is unloadable: an
unloadable library exports a special unload procedure.
The name of the unload procedure is determined by
packageName and whether or not the target interpreter is
a safe one. For normal interpreters the name of the
initialization procedure will have the form pkg\_Unload,
where pkg is the same as packageName except that the
first letter is converted to upper case and all other
letters are converted to lower case. For example, if
packageName is foo or FOo, the initialization procedure's
name will be Foo\_Unload. If the target interpreter is a
safe interpreter, then the name of the initialization
procedure will be pkg\_SafeUnload instead of pkg\_Unload.
If unload determines that a library is not unloadable \(or
unload functionality has been disabled during
compilation\), an error will be returned. If the library
is unloadable, then unload will call the unload
procedure. If the unload procedure returns TCL\_OK, unload
will proceed and decrease the proper reference count
\(depending on the target interpreter type\). When both
reference counts have reached 0, the library will be
detached from the process.
The unload procedure must match the following prototype:
typedef int Tcl\_PackageUnloadProc\(Tcl\_Interp \*interp, int
flags\);
The interp argument identifies the interpreter from which
the library is to be unloaded. The unload procedure must
return TCL\_OK or TCL\_ERROR to indicate whether or not it
completed successfully; in the event of an error it
should set the interpreter's result to point to an error
message. In this case, the result of the unload command
will be the result returned by the unload procedure. The
flags argument can be either
TCL\_UNLOAD\_DETACH\_FROM\_INTERPRETER or
TCL\_UNLOAD\_DETACH\_FROM\_PROCESS. In case the library will
remain attached to the process after the unload procedure
returns \(i.e. because the library is used by other
interpreters\), TCL\_UNLOAD\_DETACH\_FROM\_INTERPRETER will be
defined. However, if the library is used only by the
target interpreter and the library will be detached from
the application as soon as the unload procedure returns,
the flags argument will be set to
TCL\_UNLOAD\_DETACH\_FROM\_PROCESS.
The unload command cannot unload libraries that are
statically linked with the application. If fileName is an
empty string, then packageName must be specified.
If packageName is omitted or specified as an empty
string, Tcl tries to guess the name of the package. This
may be done differently on different platforms. The
default guess, which is used on most UNIX platforms, is
to take the last element of fileName, strip off the first
three characters if they are lib, and use any following
alphabetic and underline characters as the module name.
For example, the command unload libxyz4.2.so uses the
module name xyz and the command unload bin/last.so \{\}
uses the module name last.
PORTABILITY ISSUES
Unix
Not all unix operating systems support library
unloading. Under such an operating system unload
returns an error \(unless -nocomplain has been
specified\).
Macintosh
<Somebody to comment on this?>
BUGS
If the same file is loaded by different fileNames, it
will be loaded into the process's address space multiple
times. The behavior of this varies from system to system
\(some systems may detect the redundant loads, others may
not\). In case a library has been silently detached by the
operating system \(and as a result Tcl thinks the library
is still loaded\), it may be dangerous to use unload on
such a library \(as the library will be completely
detached from the application while some interpreters
will continue to use it\).
SEE ALSO
info sharedlibextension, load, safe
KEYWORDS
binary code, unloading, safe interpreter, shared library

TIP: 101Title: Export Tcltest Configuration
Version: $Revision: 1.4 $
Author: Don Porter <dgp@users.sf.net>
State: Final
Type: Project
Vote: Done
Created: 11-Jun-2002
Post-History:
Tcl-Version: 8.4
~ Abstract
Proposes public command ''tcltest::configure'' to give programmatic
control to processing configuration options of the tcltest package.
~ Rationale
During ''package require tcltest'', the internal command
''ProcessCmdLineArgs'' is evaluated. This command uses the contents
of ''$::argv'' as option-value pairs to configure several aspects of
the tcltest package.
This approach leaves two aspect of package configuration hardwired,
and outside of the control of users of ''tcltest''. First, the timing
of configuration is fixed to package load time. Second, the source of
configuration data is fixed to be the global variable ''argv''.
It would improve flexible use of tcltest to export a public command,
''tcltest::configure'', that will allow configuration of ''tcltest''
by its users at any time and from any source.
~ Proposal
Add and export the command ''tcltest::configure'', with the syntax:
| tcltest::configure ?option? ?value option value ...?
With no options, ''configure'' returns a list of the available
configurable options. With a single ''option'' argument,
''configure'' returns the corresponding value of that option, or an
error if no such option exists. In the most general form,
''configure'' accepts an even number of arguments that are alternating
options and values, and sets each option to each value.
The list of options and acceptable values are to be the same as those
currently recognized by ''tcltest'' as its command line options. The
difference is that this configuration can now be performed
programmatically, not only on the command line.
With complete programmatic access to ''tcltest'' configuration made
available, the special customization hooks ''processCmdLineArgsHook''
and ''processCmdLineArgsAddFlagsHook'' will be deprecated and removed
from the documentation. Compatibility support for their existing use
will be provided as described below.
~ Compatibility
Many existing test suites have been written depending on the
auto-configuration from ''$::argv'' at package load time. Some of
them may also be using the special customization hooks that allow the
addition of more command line options. For compatibility, if the
presence of any of these hooks is detected, ''tcltest'' will fall back
to performing its load-time configuration. Also, if any command that
can be influenced by configured values is called prior to any call to
''configure'', then automatic configuration from ''::argv'' will be
performed.
~ Copyright
This document has been placed in the public domain.

# TIP 101: Export Tcltest Configuration
Author: Don Porter <dgp@users.sf.net>
State: Final
Type: Project
Vote: Done
Created: 11-Jun-2002
Post-History:
Tcl-Version: 8.4
-----# Abstract
Proposes public command _tcltest::configure_ to give programmatic
control to processing configuration options of the tcltest package.
# Rationale
During _package require tcltest_, the internal command
_ProcessCmdLineArgs_ is evaluated. This command uses the contents
of _$::argv_ as option-value pairs to configure several aspects of
the tcltest package.
This approach leaves two aspect of package configuration hardwired,
and outside of the control of users of _tcltest_. First, the timing
of configuration is fixed to package load time. Second, the source of
configuration data is fixed to be the global variable _argv_.
It would improve flexible use of tcltest to export a public command,
_tcltest::configure_, that will allow configuration of _tcltest_
by its users at any time and from any source.
# Proposal
Add and export the command _tcltest::configure_, with the syntax:
tcltest::configure ?option? ?value option value ...?
With no options, _configure_ returns a list of the available
configurable options. With a single _option_ argument,
_configure_ returns the corresponding value of that option, or an
error if no such option exists. In the most general form,
_configure_ accepts an even number of arguments that are alternating
options and values, and sets each option to each value.
The list of options and acceptable values are to be the same as those
currently recognized by _tcltest_ as its command line options. The
difference is that this configuration can now be performed
programmatically, not only on the command line.
With complete programmatic access to _tcltest_ configuration made
available, the special customization hooks _processCmdLineArgsHook_
and _processCmdLineArgsAddFlagsHook_ will be deprecated and removed
from the documentation. Compatibility support for their existing use
will be provided as described below.
# Compatibility
Many existing test suites have been written depending on the
auto-configuration from _$::argv_ at package load time. Some of
them may also be using the special customization hooks that allow the
addition of more command line options. For compatibility, if the
presence of any of these hooks is detected, _tcltest_ will fall back
to performing its load-time configuration. Also, if any command that
can be influenced by configured values is called prior to any call to
_configure_, then automatic configuration from _::argv_ will be
performed.
# Copyright
This document has been placed in the public domain.

TIP: 102Title: Change [trace list] to [trace info]
Author: Reinhard Max <max@suse.de>
Type: Project
Tcl-Version: 8.4
State: Final
Created: 12-Jun-2002
Keywords: trace, info, introspection
Version: $Revision: 1.4 $
Vote: Done
Post-History:
~ Abstract
This TIP proposes to change the name of the introspection subcommand
of the ''trace'' command from ''list'' to ''info''.~ Rationale
Although the functionality of the ''trace'' command (as extended by
[62]) is good, the name of the introspection subcommand, ''list'' is
not such a good choice:
* The name ''info'' is already well known for introspection purposes
in Tcl (e.g. [[info]] and [[after info]].)
* The name [[trace list]] could be misunderstood as having to do with
tracing lists.
* The introspection subcommand to trace could be extended to allow
more specific queries along the lines of [[info commands]], [[info
procs]], and [[info vars]]. ''(This is outside the scope of this
TIP.)''
Hence, this TIP calls for the ''list'' subcommand to be renamed to
''info''. (Note that this also makes the subcommand for introspecting
on variable traces more similar to its old form ''vinfo''.)
~ Copyright
This document is placed in the public domain.

# TIP 102: Change [trace list] to [trace info]
Author: Reinhard Max <max@suse.de>
Type: Project
Tcl-Version: 8.4
State: Final
Created: 12-Jun-2002
Keywords: trace, info, introspection
Vote: Done
Post-History:
-----# Abstract
This TIP proposes to change the name of the introspection subcommand
of the _trace_ command from _list_ to _info_.# Rationale
Although the functionality of the _trace_ command \(as extended by
[[62]](62.md)\) is good, the name of the introspection subcommand, _list_ is
not such a good choice:
* The name _info_ is already well known for introspection purposes
in Tcl \(e.g. [info] and [after info].\)
* The name [trace list] could be misunderstood as having to do with
tracing lists.
* The introspection subcommand to trace could be extended to allow
more specific queries along the lines of [info commands], [info
procs], and [info vars]. _\(This is outside the scope of this
TIP.\)_
Hence, this TIP calls for the _list_ subcommand to be renamed to
_info_. \(Note that this also makes the subcommand for introspecting
on variable traces more similar to its old form _vinfo_.\)
# Copyright
This document is placed in the public domain.

TIP: 103Title: Argument Expansion Command
Version: $Revision: 1.13 $
Author: Peter Spjuth <peter.spjuth@space.se>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Author: Andreas Leitgeb <avl@logic.at>
State: Rejected
Type: Project
Vote: Done
Created: 15-Jun-2002
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes to add a command that can perform argument expansion
in a safe and efficient manner.
~ Introduction
Many commands take a variable number of arguments and often you find
yourself with those arguments in a list. This list must then be
expanded into individual arguments to the command. This is currently
done with eval:
|eval destroy [winfo children .]
This is a bit obscure and also very error prone when the command
becomes more complex. It is also inefficient and not object safe, why
a command specialised in doing this would be better.
~ Rationale
There have been suggestions of introducing some new syntax to Tcl to
handle argument expansion. That is a big and controversial step, and
not anything this TIP wants to meddle in. A command can improve every
point where eval has shortcomings and thus give a good result with
less means. It can also serve as a bridge to a future global syntax.
Such a command can be done in several ways and below the choice in
this TIP's specification is defended.
As examples three statements are used which will be repeated for
different alternatives. This is the eval version:
|eval destroy [winfo children .]
|eval button .b $stdargs -text \$mytext -bd $border
|eval exec \$prog $opts1 [getMoreopts] \$file1 \$file2
The eval version would be even more complex if the lists that are to
be expanded are not known to be pure. To be really safe the last
would be:
|eval exec \$prog [lrange $opts1 0 end] [lrange [getMoreopts] 0 end] \$file1 \$file2
With the proposed command they become:
|expand { destroy `[winfo children .] }
|expand { button .b `$stdargs -text $mytext -bd $border }
|expand { exec $prog `$opts1 `[getMoreopts] $file1 $file2 }
An alternative to having a local syntax is to point at the arguments
that should be expanded, either by index:
|expand {end} destroy [winfo children .]
|expand {2} button .b $stdargs -text $mytext -bd $border
|expand {2 3} exec $prog $opts1 [getMoreopts] $file1 $file2
Or by some flag mechanism:
|expand destroy + [winfo children .]
|expand button .b + $stdargs -text - $mytext -bd $border
|expand exec - $prog + $opts1 + [getMoreopts] - $file1 - $file2
Those lack in writability/readability/maintainability in a disturbing
manner.
For the choice of local syntax one goal is that it should not
violate Tcl's rules, which simplifies implementation since Tcl's
parser can do the job.
Any char that fulfils that could be used but the choice fell on ` for
forward compatibility reasons. See below.
An alternative syntax could be using enclosing `` or some
other enclosing construct like:
|expand { destroy <[winfo children .]> }
|expand { button .b <$stdargs> -text $mytext -bd $border }
|expand { exec $prog <$opts1> <[getMoreopts]> $file1 $file2 }
Paired characters are good for delimiting things. Here is the
beginning; here is the end. But this is not about a new way to
delimit things. It is about indicating a boolean choice: expand
or do not expand a word into multiple words. Whatever character
is chosen to be that indicator, it should be a single, leading
one. No pairs.
In the specification a restrictive rule was chosen that makes
it an error to use ` in a way that do not fit. This is to make
it easier to change things in the future should ideas come up
for new features. E.g., should this become a global syntax
in Tcl 9.0 it can be chosen a bit differently and be backward
compatible with the expand command.
~ Specification
A new command "expand" is added. It takes one argument, which
contains a Tcl script consisting of one command. The script may
contain comments but only one command is permitted.
The command is processed in the following manner:
1. Parse into words according to Tcl's standard rules.
2. Any word starting with ` must be followed by a single variable
or command substitution. The word is remembered and the ` is
removed.
3. Perform Tcl's normal execution steps on the new line up to the
point where the command should have been called.
4. Expand the arguments that should be expanded.
5. Execute the command.
The return value of expand is the return value of the command.
''Note 1:'' A word should really start with ` to trigger expansion
which means that words like these are not expanded:
|cmd "`$temp" \`[something]
''Note 2:'' Expansion is only performed with words like:
|cmd `$var `[somecmd $arg] `$arr([cmd $arg])
Words like these are a syntax error:
|cmd `word` `$x,$y `[foo]xy[apa]
~ Forward compatibility
One aspect of choosing a syntax here is to think about the
future. Should there later be a wish for a global syntax for argument
expansion it would be nice if it were the same as the one chosen in
the expand command. If an agreement
can be made for what may be acceptable in the future, this should
affect the specification in this TIP.
If a single character like ` is chosen for a global expand syntax
it means a backwards compatibility break. So, what chars
are likely to be used by people and thus causing problems or confusion
when backwards compatibility is broken?
Some food for thought about different chars:
|_ # Word char
|: # Gets ugly with namespace qualifiers: :$::var
|! if !$var {...}
|* string match *$suffix $line
|^ regexp ^$prefix $line
|~ cd ~$user
|| open |$prog
|. button .$w ; glob -nocomplain .$str
|= wm geometry .e =$geo
|@ .x conf -bitmap @$bmp -cursor @$cur
|< bind . <$left> ; set html <$tag>
|( expr ($a + $b) * $c ;# Confuses paren-matching
|) # Odd enough as opening, but would confuse any paren-matching
|+ expr $a +$b ;# Same for any operator
|- |%|& |?|/ open /$path
|' # Makes more sense as enclosing?
|` # Makes more sense as enclosing?
|> exec foobar >/some/file
|, append recipients ,[join $header($ccL) ,]
|{} completely forward-compatible, as {} currently cannot be
| trailed by anything but whitespace. (This would limit the
| originally proposed global syntax change to the argument
| of the expand command)
Example usage of those that seem reasonable:
|expand { exec $prog '$opts1 '[getMoreopts] $file1 $file2 }
|expand { exec $prog `$opts1 `[getMoreopts] $file1 $file2 }
|expand { exec $prog ,$opts1 ,[getMoreopts] $file1 $file2 }
For comparison, the syntax that has been proposed earlier that
would not break backwards compatibility:
|expand { exec $prog {}$opts1 {}[getMoreopts] $file1 $file2 }
|expand { exec $prog {expand}$opts1 {expand}[getMoreopts] $file1 $file2 }
~ Discussion
When first issued the TIP caused some discussion on c.l.t. Until a
summary is made, here is the thread:
http://groups.google.com/groups?th=9e77d5836b06ab1b
Another thread about it:
http://groups.google.com/groups?th=2ba287be87c2678c
~ Reference Implementation
Patch #570201
http://sourceforge.net/tracker/index.php?func=detail&aid=570201&group_id=10894&atid=310894~ Copyright
This document has been placed in the public domain.

# TIP 103: Argument Expansion Command
Author: Peter Spjuth <peter.spjuth@space.se>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Author: Andreas Leitgeb <avl@logic.at>
State: Rejected
Type: Project
Vote: Done
Created: 15-Jun-2002
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes to add a command that can perform argument expansion
in a safe and efficient manner.
# Introduction
Many commands take a variable number of arguments and often you find
yourself with those arguments in a list. This list must then be
expanded into individual arguments to the command. This is currently
done with eval:
eval destroy [winfo children .]
This is a bit obscure and also very error prone when the command
becomes more complex. It is also inefficient and not object safe, why
a command specialised in doing this would be better.
# Rationale
There have been suggestions of introducing some new syntax to Tcl to
handle argument expansion. That is a big and controversial step, and
not anything this TIP wants to meddle in. A command can improve every
point where eval has shortcomings and thus give a good result with
less means. It can also serve as a bridge to a future global syntax.
Such a command can be done in several ways and below the choice in
this TIP's specification is defended.
As examples three statements are used which will be repeated for
different alternatives. This is the eval version:
eval destroy [winfo children .]
eval button .b $stdargs -text \$mytext -bd $border
eval exec \$prog $opts1 [getMoreopts] \$file1 \$file2
The eval version would be even more complex if the lists that are to
be expanded are not known to be pure. To be really safe the last
would be:
eval exec \$prog [lrange $opts1 0 end] [lrange [getMoreopts] 0 end] \$file1 \$file2
With the proposed command they become:
expand { destroy `[winfo children .] }
expand { button .b `$stdargs -text $mytext -bd $border }
expand { exec $prog `$opts1 `[getMoreopts] $file1 $file2 }
An alternative to having a local syntax is to point at the arguments
that should be expanded, either by index:
expand {end} destroy [winfo children .]
expand {2} button .b $stdargs -text $mytext -bd $border
expand {2 3} exec $prog $opts1 [getMoreopts] $file1 $file2
Or by some flag mechanism:
expand destroy + [winfo children .]
expand button .b + $stdargs -text - $mytext -bd $border
expand exec - $prog + $opts1 + [getMoreopts] - $file1 - $file2
Those lack in writability/readability/maintainability in a disturbing
manner.
For the choice of local syntax one goal is that it should not
violate Tcl's rules, which simplifies implementation since Tcl's
parser can do the job.
Any char that fulfils that could be used but the choice fell on \` for
forward compatibility reasons. See below.
An alternative syntax could be using enclosing \`\` or some
other enclosing construct like:
expand { destroy <[winfo children .]> }
expand { button .b <$stdargs> -text $mytext -bd $border }
expand { exec $prog <$opts1> <[getMoreopts]> $file1 $file2 }
Paired characters are good for delimiting things. Here is the
beginning; here is the end. But this is not about a new way to
delimit things. It is about indicating a boolean choice: expand
or do not expand a word into multiple words. Whatever character
is chosen to be that indicator, it should be a single, leading
one. No pairs.
In the specification a restrictive rule was chosen that makes
it an error to use \` in a way that do not fit. This is to make
it easier to change things in the future should ideas come up
for new features. E.g., should this become a global syntax
in Tcl 9.0 it can be chosen a bit differently and be backward
compatible with the expand command.
# Specification
A new command "expand" is added. It takes one argument, which
contains a Tcl script consisting of one command. The script may
contain comments but only one command is permitted.
The command is processed in the following manner:
1. Parse into words according to Tcl's standard rules.
2. Any word starting with \` must be followed by a single variable
or command substitution. The word is remembered and the \` is
removed.
3. Perform Tcl's normal execution steps on the new line up to the
point where the command should have been called.
4. Expand the arguments that should be expanded.
5. Execute the command.
The return value of expand is the return value of the command.
_Note 1:_ A word should really start with \` to trigger expansion
which means that words like these are not expanded:
cmd "`$temp" \`[something]
_Note 2:_ Expansion is only performed with words like:
cmd `$var `[somecmd $arg] `$arr([cmd $arg])
Words like these are a syntax error:
cmd `word` `$x,$y `[foo]xy[apa]
# Forward compatibility
One aspect of choosing a syntax here is to think about the
future. Should there later be a wish for a global syntax for argument
expansion it would be nice if it were the same as the one chosen in
the expand command. If an agreement
can be made for what may be acceptable in the future, this should
affect the specification in this TIP.
If a single character like \` is chosen for a global expand syntax
it means a backwards compatibility break. So, what chars
are likely to be used by people and thus causing problems or confusion
when backwards compatibility is broken?
Some food for thought about different chars:
_ # Word char
: # Gets ugly with namespace qualifiers: :$::var
! if !$var {...}
* string match *$suffix $line
^ regexp ^$prefix $line
~ cd ~$user
| open |$prog
. button .$w ; glob -nocomplain .$str
= wm geometry .e =$geo
@ .x conf -bitmap @$bmp -cursor @$cur
< bind . <$left> ; set html <$tag>
( expr ($a + $b) * $c ;# Confuses paren-matching
) # Odd enough as opening, but would confuse any paren-matching
+ expr $a +$b ;# Same for any operator
- % & ?/ open /$path
' # Makes more sense as enclosing?
` # Makes more sense as enclosing?
> exec foobar >/some/file
, append recipients ,[join $header($ccL) ,]
{} completely forward-compatible, as {} currently cannot be
trailed by anything but whitespace. (This would limit the
originally proposed global syntax change to the argument
of the expand command)
Example usage of those that seem reasonable:
expand { exec $prog '$opts1 '[getMoreopts] $file1 $file2 }
expand { exec $prog `$opts1 `[getMoreopts] $file1 $file2 }
expand { exec $prog ,$opts1 ,[getMoreopts] $file1 $file2 }
For comparison, the syntax that has been proposed earlier that
would not break backwards compatibility:
expand { exec $prog {}$opts1 {}[getMoreopts] $file1 $file2 }
expand { exec $prog {expand}$opts1 {expand}[getMoreopts] $file1 $file2 }
# Discussion
When first issued the TIP caused some discussion on c.l.t. Until a
summary is made, here is the thread:
<http://groups.google.com/groups?th=9e77d5836b06ab1b>
Another thread about it:
<http://groups.google.com/groups?th=2ba287be87c2678c># Reference Implementation
Patch \#570201
<http://sourceforge.net/tracker/index.php?func=detail&aid=570201&group\_id=10894&atid=310894># Copyright
This document has been placed in the public domain.

TIP: 104Title: Generalization of the Tk Undo Subsystem
Version: $Revision: 1.6 $
Author: Ludwig Callewaert <ludwig.callewaert@belgacom.net>
Author: Larry W. Virden. <lvirden@yahoo.com>
State: Final
Type: Project
Vote: Done
Created: 19-Jun-2002
Post-History:
Discussions-To: news:comp.lang.tcl
Tcl-Version: 8.4
~ Abstract
This TIP proposes a reimplementation of the Tk Text widget undo
feature. The text widget interface is not affected. No functional
changes are made at the Tcl level. The purpose of the
reimplementation is to move the undo feature from a text only
implementation to a general implementation also usable by other
widgets. This opens the door to undoing also tag, mark and other
operations, and allows for an exposure of the undo stack at the Tcl
level. These new features are however not part of this TIP.
~ Rationale
As stated in the abstract, the current implementation of the text
widget undo feature only allows for text changes to be undone. The
usefulness of the undo feature would increase tremendously if
other operations could be undone (tags, marks, embedded windows, ...).
This was already part of the [26] discussions. This TIP deals with
the generalization of the undo stack to cope with these requirements.
~ Specification
As the undo functionality is no longer text widget specific, it has
been put in a separate file ''generic/tkUndo.c'' along with its header
file ''generic/tkUndo.h''. The ''TkUndoRedoStack'' is a structure
containing the undo and redo stacks. These undo and redo stacks are
linked lists of ''TkUndoAtom'' structures. There are two types of
these atoms: the separator (similar to the previous implementation)
and the command. When the type is command, both an ''apply'' and a
''revert'' action need to be provided. The apply action is for the
redo. The revert action is for the undo. Both are pointers to a
''Tcl_Obj'', so they can (and should) contain a Tcl script.
The following functions all operating on a ''TkUndoRedoStack''
stack are provided to implement the undo/redo functionality.
1. ''TkUndoInitStack(interp)'': returns a pointer to an initialized
TkUndoRedoStack and stores ''interp'' in that stack for script
evaluation.
2. ''TkUndoClearStacks(stack)'': clears both the undo and the redo
stacks.
3. ''TkUndoFreeStack(stack)'': clears both undo and redo stacks and
frees any memory allocated to ''stack''.
4. ''TkUndoInsertUndoSeparator(stack)'': inserts a
separator on the undo stack. Note that there is currently no
need for a ''TkUndoInsertRedoSeparator'' function, as the redo
stack is managed by the internals of ''TkUndo''.
5. ''TkUndoPushAction(stack, actionScript, revertScript)'': pushes
an action of the undo stack (an atom of type command).
''actionScript'' and ''revertScript'' are ''Tcl_DString''
pointers that provide the script to redo and undo the action
respectively. The redo stack is cleared.
6. ''TkUndoRevert(stack)'': undo a compound action.
Compound means all revert script of action between two
separators on the undo stack are evaluated in the stack's
interpreter and the actions are moved to the redo stack.
Returns TCL_ERROR when unsuccessful (stack empty for instance),
and TCL_OK otherwise.
7. ''TkUndoApply(stack)'': redo a compound action. The
apply script of all actions between two separators on the redo
stack is evaluated in the stack's interpreter. The actions are
moved to the undo stack. Returns TCL_ERROR when unsuccessful
(stack empty for instance), and TCL_OK otherwise.
8. ''TkUndoSetDepth(stack, maxDepth)'': sets the maximum number
of compound actions stored on the stack to ''maxDepth''. By
default, stacks are unlimited, and a value of ''maxDepth'' <= 0
resets the stack to be unlimited.
9. The option ''-maxundo'' is added to the text widget to access the
stack limit feature of the text widget's undo stack from the
script level.
These functions are sufficient to implement the current undo
functionality of the text widget, and they have been used for this
purpose.
~ Reference Implementation
See patch #554763 on SourceForge: ''
https://sourceforge.net/tracker/?func=detail&atid=312997&aid=554763&group_id=12997''~ Copyright
This document has been placed in the public domain.

# TIP 104: Generalization of the Tk Undo Subsystem
Author: Ludwig Callewaert <ludwig.callewaert@belgacom.net>
Author: Larry W. Virden. <lvirden@yahoo.com>
State: Final
Type: Project
Vote: Done
Created: 19-Jun-2002
Post-History:
Discussions-To: news:comp.lang.tcl
Tcl-Version: 8.4
-----# Abstract
This TIP proposes a reimplementation of the Tk Text widget undo
feature. The text widget interface is not affected. No functional
changes are made at the Tcl level. The purpose of the
reimplementation is to move the undo feature from a text only
implementation to a general implementation also usable by other
widgets. This opens the door to undoing also tag, mark and other
operations, and allows for an exposure of the undo stack at the Tcl
level. These new features are however not part of this TIP.
# Rationale
As stated in the abstract, the current implementation of the text
widget undo feature only allows for text changes to be undone. The
usefulness of the undo feature would increase tremendously if
other operations could be undone \(tags, marks, embedded windows, ...\).
This was already part of the [[26]](26.md) discussions. This TIP deals with
the generalization of the undo stack to cope with these requirements.
# Specification
As the undo functionality is no longer text widget specific, it has
been put in a separate file _generic/tkUndo.c_ along with its header
file _generic/tkUndo.h_. The _TkUndoRedoStack_ is a structure
containing the undo and redo stacks. These undo and redo stacks are
linked lists of _TkUndoAtom_ structures. There are two types of
these atoms: the separator \(similar to the previous implementation\)
and the command. When the type is command, both an _apply_ and a
_revert_ action need to be provided. The apply action is for the
redo. The revert action is for the undo. Both are pointers to a
_Tcl\_Obj_, so they can \(and should\) contain a Tcl script.
The following functions all operating on a _TkUndoRedoStack_
stack are provided to implement the undo/redo functionality.
1. _TkUndoInitStack\(interp\)_: returns a pointer to an initialized
TkUndoRedoStack and stores _interp_ in that stack for script
evaluation.
2. _TkUndoClearStacks\(stack\)_: clears both the undo and the redo
stacks.
3. _TkUndoFreeStack\(stack\)_: clears both undo and redo stacks and
frees any memory allocated to _stack_.
4. _TkUndoInsertUndoSeparator\(stack\)_: inserts a
separator on the undo stack. Note that there is currently no
need for a _TkUndoInsertRedoSeparator_ function, as the redo
stack is managed by the internals of _TkUndo_.
5. _TkUndoPushAction\(stack, actionScript, revertScript\)_: pushes
an action of the undo stack \(an atom of type command\).
_actionScript_ and _revertScript_ are _Tcl\_DString_
pointers that provide the script to redo and undo the action
respectively. The redo stack is cleared.
6. _TkUndoRevert\(stack\)_: undo a compound action.
Compound means all revert script of action between two
separators on the undo stack are evaluated in the stack's
interpreter and the actions are moved to the redo stack.
Returns TCL\_ERROR when unsuccessful \(stack empty for instance\),
and TCL\_OK otherwise.
7. _TkUndoApply\(stack\)_: redo a compound action. The
apply script of all actions between two separators on the redo
stack is evaluated in the stack's interpreter. The actions are
moved to the undo stack. Returns TCL\_ERROR when unsuccessful
\(stack empty for instance\), and TCL\_OK otherwise.
8. _TkUndoSetDepth\(stack, maxDepth\)_: sets the maximum number
of compound actions stored on the stack to _maxDepth_. By
default, stacks are unlimited, and a value of _maxDepth_ <= 0
resets the stack to be unlimited.
9. The option _-maxundo_ is added to the text widget to access the
stack limit feature of the text widget's undo stack from the
script level.
These functions are sufficient to implement the current undo
functionality of the text widget, and they have been used for this
purpose.
# Reference Implementation
See patch \#554763 on SourceForge: _<https://sourceforge.net/tracker/?func=detail&atid=312997&aid=554763&group\_id=12997>_# Copyright
This document has been placed in the public domain.

TIP: 105Title: Add Prefix Matching for Switch
State: Withdrawn
Type: Project
Tcl-Version: 8.5
Vote: Pending
Post-History:
Version: $Revision: 1.4 $
Author: Donal K. Fellows <fellowsd@cs.man.ac.uk>
Created: 03-Jul-2002
Obsoleted-By: 195
~ Abstract
This TIP adds a new option to the [[switch]] command to support
matching of strings to unique prefixes of patterns, similar to Tcl's
existing subcommand-name matching or Tk's option-name matching.
~ Rationale
When code (particularly in script libraries) wants to support shortest
unique prefix matching in the manner of the Tcl core (as provided by
''Tcl_GetIndexFromObj'') currently either the prefixes have to be
precomputed (by hand or by script) or the matching has to be done
backwards. In the first case, this is either error-prone or requires
an extra piece of code that has to be developed by the programmer. In
the second case, the code has to be converted into a pattern which is
matched against the list of supported options in some way, which is
either inefficient or has hazards if the string being matched contains
characters that are meaningful to the matching engine being used.
Instead, it would be far nicer if we could make the core support this
directly, so that script authors could just say what they mean.
~ Proposed Change
To support this, I propose modifying the ''switch'' command to take an
extra option ''-prefix'' (which should be mutually exclusive with
''-exact'', ''-glob'' and ''-regexp'' of course) to enable prefix
matching. When prefix matching is enabled, the arm chosen for
execution will be the one such that the switch value is identical to
or an unambiguous prefix of its pattern (i.e. it will not be a prefix
of any other pattern listed, unless the pattern of the arm chosen is
exactly equal to the switch value.) If there is no arm whose pattern
is an unambiguous prefix of the switch value, the default arm will be
selected for execution, or if there is no default arm, the switch
command will terminate without an error and with an empty result (this
is in contrast to the behaviour of ''Tcl_GetIndexFromObj''.)
~ Examples
The command:
|switch -prefix f {
| foo {
| puts "matched foo"
| }| bar {
| puts "matched bar"
| }|}
prints "matched foo". The command:
|switch -prefix b {
| bar {
| puts "matched bar"
| }| boo {
| puts "matched boo"
| }| default {
| puts "the default action"
| }|}
prints "the default action" ("b" is a prefix of two patterns.) The
command:
|switch -prefix tcl {
| tcl {
| puts "The Tool Command Language"
| }| tk {
| puts "The Tk Toolkit"
| }| tcl/tk {
| puts "A cool combination"
| }|}
prints "The Tool Command Language" (although "tcl" is a prefix of two
patterns, it matches one of them exactly.)
~ Copyright
This document has been placed in the public domain.

# TIP 105: Add Prefix Matching for Switch
State: Withdrawn
Type: Project
Tcl-Version: 8.5
Vote: Pending
Post-History:
Author: Donal K. Fellows <fellowsd@cs.man.ac.uk>
Created: 03-Jul-2002
Obsoleted-By: 195
-----# Abstract
This TIP adds a new option to the [switch] command to support
matching of strings to unique prefixes of patterns, similar to Tcl's
existing subcommand-name matching or Tk's option-name matching.
# Rationale
When code \(particularly in script libraries\) wants to support shortest
unique prefix matching in the manner of the Tcl core \(as provided by
_Tcl\_GetIndexFromObj_\) currently either the prefixes have to be
precomputed \(by hand or by script\) or the matching has to be done
backwards. In the first case, this is either error-prone or requires
an extra piece of code that has to be developed by the programmer. In
the second case, the code has to be converted into a pattern which is
matched against the list of supported options in some way, which is
either inefficient or has hazards if the string being matched contains
characters that are meaningful to the matching engine being used.
Instead, it would be far nicer if we could make the core support this
directly, so that script authors could just say what they mean.
# Proposed Change
To support this, I propose modifying the _switch_ command to take an
extra option _-prefix_ \(which should be mutually exclusive with
_-exact_, _-glob_ and _-regexp_ of course\) to enable prefix
matching. When prefix matching is enabled, the arm chosen for
execution will be the one such that the switch value is identical to
or an unambiguous prefix of its pattern \(i.e. it will not be a prefix
of any other pattern listed, unless the pattern of the arm chosen is
exactly equal to the switch value.\) If there is no arm whose pattern
is an unambiguous prefix of the switch value, the default arm will be
selected for execution, or if there is no default arm, the switch
command will terminate without an error and with an empty result \(this
is in contrast to the behaviour of _Tcl\_GetIndexFromObj_.\)
# Examples
The command:
switch -prefix f {
foo {
puts "matched foo"
} bar {
puts "matched bar"
} }
prints "matched foo". The command:
switch -prefix b {
bar {
puts "matched bar"
} boo {
puts "matched boo"
} default {
puts "the default action"
} }
prints "the default action" \("b" is a prefix of two patterns.\) The
command:
switch -prefix tcl {
tcl {
puts "The Tool Command Language"
} tk {
puts "The Tk Toolkit"
} tcl/tk {
puts "A cool combination"
} }
prints "The Tool Command Language" \(although "tcl" is a prefix of two
patterns, it matches one of them exactly.\)
# Copyright
This document has been placed in the public domain.

TIP: 106Title: Add Encoding Abilities to the [dde] Command
Version: $Revision: 1.14 $
Author: Harald Oehlmann <harald.oehlmann@elmicron.de>
State: Final
Type: Project
Vote: Done
Created: 13-Aug-2002
Post-History:
Tcl-Version: 8.6
~ Abstract
When using Windows DDE communication with non-Tcl programs, the encoding of
the exchanged strings is mostly needed to be the system encoding. Selection
of this behaviour should be possible with in the '''dde''' command should be
done by a parameter.
~ Specification
Extend the '''dde''' commands taking a data argument by the switch
'''-binary''':
> '''dde execute''' ?'''-async'''? ?'''-binary'''? ''service topic data''
> '''dde poke''' ?'''-binary'''? ''service topic item data''
The argument ''data'' is taken as a binary string if the '''-binary''' switch
is given. Otherwise, it is interpreted as utf-8.
Examples:
| dde execute -binary CS CS [encoding convertto [encoding system] Äpfel]\0
| dde poke -binary CS CS I [encoding convertto [encoding system] Äpfel]\0
~ Rationale
The communication with DDE with external programs uses the format clipboard
''CF_TEXT'' and the sent text should be coded in the system encoding
(''cp1252'' in my locale).
Most people who use DDE to communicate with, for example, Excel use the fact
that what Excel expects (''cp1252'') and what Tcl actually sends (''utf-8'')
is identical for 7-bit values and they don't use 8-bit values. Unfortunately,
characters used in languages like German, French, etc., are located over this
limit and thus are not transferable.
Peter Hardie addressed this point on 2000-10-26 in the Tcl Feature Request at
SourceForge (#219185: "dde only handles UTF-8 data (-binary patch available)"
[http://sf.net/tracker/?func=detail&aid=219185&group_id=10894&atid=360894]).
His proposal was to add a '''-binary''' option.
This is a reasonable solution, because any encoding including the system
encoding may be used as shown in the upper example.
~ Reference Implementation
See the ''tip-106-impl'' branch in Tcl's fossil repository
[https://core.tcl.tk/tcl/timeline?r=tip-106-impl].
~ Rejected Alternatives
I proposed to use a switch ?'''-encoding''' ''encoding''? which would
avoid the preparation of an encoded string by '''encoding convertto'''.
DDE is so little used at those days so a minimal support is sufficient.
The '''dde request''' subcommand already has a parameter ''-binary''.
It is more logical to extend this to the other commands.
~ Copyright
This document has been placed in the public domain

# TIP 106: Add Encoding Abilities to the [dde] Command
Author: Harald Oehlmann <harald.oehlmann@elmicron.de>
State: Final
Type: Project
Vote: Done
Created: 13-Aug-2002
Post-History:
Tcl-Version: 8.6
-----# Abstract
When using Windows DDE communication with non-Tcl programs, the encoding of
the exchanged strings is mostly needed to be the system encoding. Selection
of this behaviour should be possible with in the **dde** command should be
done by a parameter.
# Specification
Extend the **dde** commands taking a data argument by the switch
**-binary**:
> **dde execute** ?**-async**? ?**-binary**? _service topic data_
> **dde poke** ?**-binary**? _service topic item data_
The argument _data_ is taken as a binary string if the **-binary** switch
is given. Otherwise, it is interpreted as utf-8.
Examples:
dde execute -binary CS CS [encoding convertto [encoding system] Ã„pfel]\0
dde poke -binary CS CS I [encoding convertto [encoding system] Ã„pfel]\0
# Rationale
The communication with DDE with external programs uses the format clipboard
_CF\_TEXT_ and the sent text should be coded in the system encoding
\(_cp1252_ in my locale\).
Most people who use DDE to communicate with, for example, Excel use the fact
that what Excel expects \(_cp1252_\) and what Tcl actually sends \(_utf-8_\)
is identical for 7-bit values and they don't use 8-bit values. Unfortunately,
characters used in languages like German, French, etc., are located over this
limit and thus are not transferable.
Peter Hardie addressed this point on 2000-10-26 in the Tcl Feature Request at
SourceForge \(\#219185: "dde only handles UTF-8 data \(-binary patch available\)"
<http://sf.net/tracker/?func=detail&aid=219185&group_id=10894&atid=360894> \).
His proposal was to add a **-binary** option.
This is a reasonable solution, because any encoding including the system
encoding may be used as shown in the upper example.
# Reference Implementation
See the _tip-106-impl_ branch in Tcl's fossil repository
<https://core.tcl.tk/tcl/timeline?r=tip-106-impl> .
# Rejected Alternatives
I proposed to use a switch ?**-encoding** _encoding_? which would
avoid the preparation of an encoded string by **encoding convertto**.
DDE is so little used at those days so a minimal support is sufficient.
The **dde request** subcommand already has a parameter _-binary_.
It is more logical to extend this to the other commands.
# Copyright
This document has been placed in the public domain

TIP: 107Title: Fix the 2-second "raise delay" in Tk
Version: $Revision: 1.4 $
Author: Joe English <jenglish@flightlab.com>
State: Final
Type: Project
Created: 28-Aug-2002
Tcl-Version: 8.4
Vote: Done
Post-History:
~ Abstract
This TIP explains the genesis of the long delays often associated with
the [[raise]] and [[lower]] commands under Unix/X with some window
managers, as well as describing the solution.
~ Rationale
Currently, Tk's [[raise]] and [[lower]] commands do not return to the
caller until the operation has completed. Under Unix, the window
manager is responsible for changing the stacking order of toplevel
windows, so [[raise]] and [[lower]] must wait for a notification from
the WM before returning. Not all window managers are ICCCM-compliant
in this regard, however, so the operation may time out instead.
This two-second "raise delay" has been a longstanding, persistent
problem in Tk. It has supposedly been fixed several times, but the
problem keeps reoccurring under new window managers and new
environments. At present, the problem is most noticeable under KDE 2
and KDE 3.
~ Proposal
Change Tk so that [[raise]] and [[lower]] return immediately, without
waiting for a notification that may not be forthcoming.
This should not be be a controversial change. This behaviour is not
documented anywhere, and is not observable by Tk programs except via
[[wm stackorder]] (see [74]).
Moreover, the guarantee is largely meaningless. After [[raise]]
returns, the window ''contents'' may still not be visible (there may
be pending <Expose> events, for example), and the actual position in
the stacking order is still subject to window manager intervention.
~ Compatibility Issues
The only Tk programs that would break with this change are ones which
expect the return value of [[wm stackorder]] to reflect the results of
any immediately-preceding [[raise]] and [[lower]] commands. (The Tk
test suite is one such program, and would need to be modified).
Unfortunately there is no reliable way to fix such programs -
[[update]] will not work, and the ICCCM does not, to the author's
knowledge, provide a way to synchronize with the window manager to
make sure it has processed all outstanding client requests. Even if
it did, this wouldn't help - the raise delay problem only occurs under
non-compliant window managers to begin with!
Since the stacking order is not observable except through [[wm
stackorder]] - that was the whole point of [74] - no other programs
will be affected. (Note that [[wm stackorder]] will still work: the
only difference is that it may return soon-to-be out-of-date
information. Since this is the case already - the user may restack or
iconify windows at any time - this change should be low-impact.)
~ Reference Implementation
See Sourceforge Tk Patch #601518.
http://sourceforge.net/tracker/index.php?func=detail&aid=601518&group_id=12997&atid=312997~ Author's Note
Could we fast-track this? It's a longstanding problem with a simple
fix and ought to make it in before 8.4 goes final.
~ Detailed Analysis
First, some terminology:
* ''toplevel'': a Tk [[toplevel]] window.
* ''wrapper'': an auxiliary window created by Tk to hold the
toplevel and its (optional) menubar. Initially created as a
child of the root window.
* ''client window'': From the window manager's perspective, any
window created as a child of the root window by an X client.
Tk wrapper windows are client windows. Most window managers
reparent client windows under a new frame window which holds
window manager decorations.
* ''reparent'': Used as a noun, the immediate parent of a
wrapper which has been reparented by a window manager.
* ''frame'': The immediate child of the root window (or virtual
root window) created by the window manager to hold a client
window and its decorations. May or may not be the same as the
reparent window.
Next, some methodology:
The correct way to change the stacking order of a client window is to
make a ''ConfigureRequest'' on the client window with ''stack_mode''
set appropriately. If the client has not been reparented, then the X
server performs the operation directly, and will send a
''ConfigureNotify'' back to the client if, and only if, the actual
stacking order has changed. (Raising a window which is already at the
top of the stacking order will not result in a ''ConfigureNotify'',
for example).
If the client window ''has'' been reparented (which is usually the
case), then the window manager intercepts the request and, at its
discretion, restacks the frame window instead. It then sends a
synthetic ''ConfigureNotify'' back to the client, regardless of
whether or not it honored the request.
If the stacking order is to be changed relative to some other window -
that is, if the ''sibling'' field is also set - and the client has
been reparented, then the ''ConfigureRequest'' will fail with a
''BadMatch'' error before it gets to the WM. Clients must be prepared
to handle this case by catching the error and re-sending a synthetic
''ConfigureRequest'' to the root window, which the WM receives and
handles as above.
See ICCCM section 4.1.5 "Configuring the Window" for the full
specification. The Xlib function ''XReconfigureWMWindow()'' takes
care of all these details.
Now, some archaeology:
Tk 4.0 did not do this: instead, it called ''XConfigureWindow()'' on
the ''reparent'' window, then waited for a ''ConfigureNotify'' on that
window.
This was wrong on at least two counts. First, the reparent window
might not be the same as the frame window, in which case this would
have no effect at all. (In 4DWm and Sawfish, for example, the
reparent window is a child of an outer frame window). Second, it's
not ICCCM-compliant (Tk doesn't own the reparent window and shouldn't
be mucking with it).
Tk 4.0 also included several heuristics that attempted to determine
when the operation was unnecessary, to avoid waiting for a
''ConfigureNotify'' on the reparent that was not forthcoming.
In Tk 4.1, the (incorrect) call to ''XConfigureWindow()'' on the
reparent was changed to a (correct) call to ''XReconfigureWMWindow()''
on the wrapper, but the old heuristic code was left mostly intact.
Browsing the CVS logs and the older Tk Changelog, we see that this
code has been updated several times to account for new conditions, but
ultimately without success: the problem persists.
These heuristics are not needed at all under WMs which send a
synthetic ''ConfigureNotify'' in response to client window stacking
order changes. On some non-compliant WMs, however, they may help
lessen the problem - more by accident than by design - if the reparent
is the same as the frame window then the Tk 4.0 heuristics sometimes
still work. But even then the heuristics are not reliable. For
instance under KDE 2.2 and KDE 3, calling [[raise]] twice in
succession always results in a 2-second delay.
It is the author's opinion that the only way forward is to let
[[raise]] and [[lower]] run asynchronously, and fix the two-second
raise delay once and for all.
~ Copyright
This document is hereby placed in the public domain.

# TIP 107: Fix the 2-second "raise delay" in Tk
Author: Joe English <jenglish@flightlab.com>
State: Final
Type: Project
Created: 28-Aug-2002
Tcl-Version: 8.4
Vote: Done
Post-History:
-----# Abstract
This TIP explains the genesis of the long delays often associated with
the [raise] and [lower] commands under Unix/X with some window
managers, as well as describing the solution.
# Rationale
Currently, Tk's [raise] and [lower] commands do not return to the
caller until the operation has completed. Under Unix, the window
manager is responsible for changing the stacking order of toplevel
windows, so [raise] and [lower] must wait for a notification from
the WM before returning. Not all window managers are ICCCM-compliant
in this regard, however, so the operation may time out instead.
This two-second "raise delay" has been a longstanding, persistent
problem in Tk. It has supposedly been fixed several times, but the
problem keeps reoccurring under new window managers and new
environments. At present, the problem is most noticeable under KDE 2
and KDE 3.
# Proposal
Change Tk so that [raise] and [lower] return immediately, without
waiting for a notification that may not be forthcoming.
This should not be be a controversial change. This behaviour is not
documented anywhere, and is not observable by Tk programs except via
[wm stackorder] \(see [[74]](74.md)\).
Moreover, the guarantee is largely meaningless. After [raise]
returns, the window _contents_ may still not be visible \(there may
be pending <Expose> events, for example\), and the actual position in
the stacking order is still subject to window manager intervention.
# Compatibility Issues
The only Tk programs that would break with this change are ones which
expect the return value of [wm stackorder] to reflect the results of
any immediately-preceding [raise] and [lower] commands. \(The Tk
test suite is one such program, and would need to be modified\).
Unfortunately there is no reliable way to fix such programs -
[update] will not work, and the ICCCM does not, to the author's
knowledge, provide a way to synchronize with the window manager to
make sure it has processed all outstanding client requests. Even if
it did, this wouldn't help - the raise delay problem only occurs under
non-compliant window managers to begin with!
Since the stacking order is not observable except through [wm
stackorder] - that was the whole point of [[74]](74.md) - no other programs
will be affected. \(Note that [wm stackorder] will still work: the
only difference is that it may return soon-to-be out-of-date
information. Since this is the case already - the user may restack or
iconify windows at any time - this change should be low-impact.\)
# Reference Implementation
See Sourceforge Tk Patch \#601518.
<http://sourceforge.net/tracker/index.php?func=detail&aid=601518&group\_id=12997&atid=312997># Author's Note
Could we fast-track this? It's a longstanding problem with a simple
fix and ought to make it in before 8.4 goes final.
# Detailed Analysis
First, some terminology:
* _toplevel_: a Tk [toplevel] window.
* _wrapper_: an auxiliary window created by Tk to hold the
toplevel and its \(optional\) menubar. Initially created as a
child of the root window.
* _client window_: From the window manager's perspective, any
window created as a child of the root window by an X client.
Tk wrapper windows are client windows. Most window managers
reparent client windows under a new frame window which holds
window manager decorations.
* _reparent_: Used as a noun, the immediate parent of a
wrapper which has been reparented by a window manager.
* _frame_: The immediate child of the root window \(or virtual
root window\) created by the window manager to hold a client
window and its decorations. May or may not be the same as the
reparent window.
Next, some methodology:
The correct way to change the stacking order of a client window is to
make a _ConfigureRequest_ on the client window with _stack\_mode_
set appropriately. If the client has not been reparented, then the X
server performs the operation directly, and will send a
_ConfigureNotify_ back to the client if, and only if, the actual
stacking order has changed. \(Raising a window which is already at the
top of the stacking order will not result in a _ConfigureNotify_,
for example\).
If the client window _has_ been reparented \(which is usually the
case\), then the window manager intercepts the request and, at its
discretion, restacks the frame window instead. It then sends a
synthetic _ConfigureNotify_ back to the client, regardless of
whether or not it honored the request.
If the stacking order is to be changed relative to some other window -
that is, if the _sibling_ field is also set - and the client has
been reparented, then the _ConfigureRequest_ will fail with a
_BadMatch_ error before it gets to the WM. Clients must be prepared
to handle this case by catching the error and re-sending a synthetic
_ConfigureRequest_ to the root window, which the WM receives and
handles as above.
See ICCCM section 4.1.5 "Configuring the Window" for the full
specification. The Xlib function _XReconfigureWMWindow\(\)_ takes
care of all these details.
Now, some archaeology:
Tk 4.0 did not do this: instead, it called _XConfigureWindow\(\)_ on
the _reparent_ window, then waited for a _ConfigureNotify_ on that
window.
This was wrong on at least two counts. First, the reparent window
might not be the same as the frame window, in which case this would
have no effect at all. \(In 4DWm and Sawfish, for example, the
reparent window is a child of an outer frame window\). Second, it's
not ICCCM-compliant \(Tk doesn't own the reparent window and shouldn't
be mucking with it\).
Tk 4.0 also included several heuristics that attempted to determine
when the operation was unnecessary, to avoid waiting for a
_ConfigureNotify_ on the reparent that was not forthcoming.
In Tk 4.1, the \(incorrect\) call to _XConfigureWindow\(\)_ on the
reparent was changed to a \(correct\) call to _XReconfigureWMWindow\(\)_
on the wrapper, but the old heuristic code was left mostly intact.
Browsing the CVS logs and the older Tk Changelog, we see that this
code has been updated several times to account for new conditions, but
ultimately without success: the problem persists.
These heuristics are not needed at all under WMs which send a
synthetic _ConfigureNotify_ in response to client window stacking
order changes. On some non-compliant WMs, however, they may help
lessen the problem - more by accident than by design - if the reparent
is the same as the frame window then the Tk 4.0 heuristics sometimes
still work. But even then the heuristics are not reliable. For
instance under KDE 2.2 and KDE 3, calling [raise] twice in
succession always results in a 2-second delay.
It is the author's opinion that the only way forward is to let
[raise] and [lower] run asynchronously, and fix the two-second
raise delay once and for all.
# Copyright
This document is hereby placed in the public domain.

TIP: 108Title: Summary of Changes to Generic Tcl/Tk Code to Enable Mac OS X Port
Version: $Revision: 1.2 $
Author: Jim Ingham <jingham@apple.com>
State: Final
Type: Informative
Vote: No voting
Created: 29-Aug-2002
Post-History:
~ Abstract
The submission of the changes to generic Tcl/Tk necessary for the Mac
OS X port was discussed on the Tcl Core mailing list. In light of the
very minor changes to shared code involved, and to facilitate
including the port in the 8.4 release, it was decided that we would
not hold a formal vote. This informational TIP is offered to
summarize the work done, however, and to maintain a more complete
record.
~ What Changes are Required for the Mac OS X Port?
These fall into two parts: macosx only changes, and changes that
effect generic code.
The Mac OS X-only changes again fall into two parts. On the one hand,
we introduced new macosx directories to the Tcl and Tk trees, at the
same level as the win, unix and mac directories. At present, the
''tcl/macosx'' directory only contains one ''.c'' file and a project
file. The ''tk/macosx'' directory is much more substantial. This set
of changes manifestly only effects this port, and since something is
better than nothing, should be uncontroversial.
The other Mac OS X-only part is the addition of Mac OS X specific
elements to the ''.decls'' files. This should also be
uncontroversial, though I had to add some slightly non-trivial code to
the ''genStubs.tcl'' file to handle the fact that Tk loosely uses
"unix" to mean "X11", which for Mac OS X is not the case.
The Tcl side of Mac OS X is clearly unix, but the same Tcl can in fact
be used with X11 (there is a native X Server in quite common use on
Mac OS X) and with the Aqua based Tk port.
In the end, however, the stubs generated for the generic, mac, win &
X11 parts of Tcl/Tk are the same, and there is just some extra logic
for the aqua part, so the result effects only Mac OS X code.
The generic code changes are quite small - a testament to the design
of the Tcl/Tk porting layers.
1. We changed the ''configure.in'', ''Makefile.in'' and ''tcl.m4'' to
handle building Tcl in a the Framework form that is common on Mac
OS X.
2. I added a bit of code (conditionalized to Mac OS X) to
''tclUnixInit.c'' (in the ''TclpSetVariables'' function) to
support looking for script files embedded in the Mac OS X
Framework bundle. This fits the Mac OS X model better than
putting files in ''/usr/local'' or such-like.
3. I added a few more elements to the ''notifierProcPtr''. For the
aqua Tk, we need to swap the Unix notifier with our own, and so we
needed more control over the notifier than was allowed. This
change has no effect if you don't use it, however.
4. I added a function, ''TkGetFirstTextLayout'', which gets the run
of text up the the first wrap. I have to get this because the Mac
OS X button control doesn't like a newline in the button text. It
is a private function, however, so it doesn't cause any
incompatibilities.
5. We had to change various places in the Tk script code and the
demos where the implicit assumption was made that [[string equal
$tcl_platform(platform) "unix"]] meant you were using X11. To
this end, we will add a ''windowingsystem'' subcommand to the
''tk'' command, and then using it to replace the cases where
''tcl_platform'' was being erroneously checked. This command will
return "x11" for an X11 server, "aqua" for the native Mac OS X
window manager, "win32" for Windows, and "classic" for Classic
MacOS.
~ Reference Implementation
The reference implementation is on the ''macosx-8-4-branch'' in the
SourceForge CVS repository.
~ Copyright
This document has been placed in the public domain.

# TIP 108: Summary of Changes to Generic Tcl/Tk Code to Enable Mac OS X Port
Author: Jim Ingham <jingham@apple.com>
State: Final
Type: Informative
Vote: No voting
Created: 29-Aug-2002
Post-History:
-----# Abstract
The submission of the changes to generic Tcl/Tk necessary for the Mac
OS X port was discussed on the Tcl Core mailing list. In light of the
very minor changes to shared code involved, and to facilitate
including the port in the 8.4 release, it was decided that we would
not hold a formal vote. This informational TIP is offered to
summarize the work done, however, and to maintain a more complete
record.
# What Changes are Required for the Mac OS X Port?
These fall into two parts: macosx only changes, and changes that
effect generic code.
The Mac OS X-only changes again fall into two parts. On the one hand,
we introduced new macosx directories to the Tcl and Tk trees, at the
same level as the win, unix and mac directories. At present, the
_tcl/macosx_ directory only contains one _.c_ file and a project
file. The _tk/macosx_ directory is much more substantial. This set
of changes manifestly only effects this port, and since something is
better than nothing, should be uncontroversial.
The other Mac OS X-only part is the addition of Mac OS X specific
elements to the _.decls_ files. This should also be
uncontroversial, though I had to add some slightly non-trivial code to
the _genStubs.tcl_ file to handle the fact that Tk loosely uses
"unix" to mean "X11", which for Mac OS X is not the case.
The Tcl side of Mac OS X is clearly unix, but the same Tcl can in fact
be used with X11 \(there is a native X Server in quite common use on
Mac OS X\) and with the Aqua based Tk port.
In the end, however, the stubs generated for the generic, mac, win &
X11 parts of Tcl/Tk are the same, and there is just some extra logic
for the aqua part, so the result effects only Mac OS X code.
The generic code changes are quite small - a testament to the design
of the Tcl/Tk porting layers.
1. We changed the _configure.in_, _Makefile.in_ and _tcl.m4_ to
handle building Tcl in a the Framework form that is common on Mac
OS X.
2. I added a bit of code \(conditionalized to Mac OS X\) to
_tclUnixInit.c_ \(in the _TclpSetVariables_ function\) to
support looking for script files embedded in the Mac OS X
Framework bundle. This fits the Mac OS X model better than
putting files in _/usr/local_ or such-like.
3. I added a few more elements to the _notifierProcPtr_. For the
aqua Tk, we need to swap the Unix notifier with our own, and so we
needed more control over the notifier than was allowed. This
change has no effect if you don't use it, however.
4. I added a function, _TkGetFirstTextLayout_, which gets the run
of text up the the first wrap. I have to get this because the Mac
OS X button control doesn't like a newline in the button text. It
is a private function, however, so it doesn't cause any
incompatibilities.
5. We had to change various places in the Tk script code and the
demos where the implicit assumption was made that [string equal
$tcl\_platform\(platform\) "unix"] meant you were using X11. To
this end, we will add a _windowingsystem_ subcommand to the
_tk_ command, and then using it to replace the cases where
_tcl\_platform_ was being erroneously checked. This command will
return "x11" for an X11 server, "aqua" for the native Mac OS X
window manager, "win32" for Windows, and "classic" for Classic
MacOS.
# Reference Implementation
The reference implementation is on the _macosx-8-4-branch_ in the
SourceForge CVS repository.
# Copyright
This document has been placed in the public domain.

TIP: 109Title: New Look for Checkbutton and Radiobutton on Unix
State: Final
Type: Project
Tcl-Version: 8.5
Vote: Done
Post-History:
Version: $Revision: 1.4 $
Author: Brian Griffin <bgriffin@model.com>
Created: 01-Oct-2002
~ Abstract
This TIP proposes changing the look of the Tk checkbutton and
radiobutton widgets on Unix to more closely match the Windows
counterparts.
~ Rationale
The current visual aspect of the Unix version checkbutton and
radiobutton has proved to be confusing to users. The distinction
between selected (On) and unselected (Off) states are not visually
different enough to clearly identify one from another. Indeed, in the
rare case where only one checkbutton is present, one cannot tell for
certain if the state is On or Off. With a check or dot mark icon
(dependent on the type of widget) displayed in the Windows version,
there is no question about the state of the widget.
~ Proposed Changes
The checkbutton shall (when the indicator is turned on) display an
check-mark or other distinguishing icon/symbol that clearly indicates
an On state. The Off state will be displayed with an empty box. The
state values will ''not'' be indicated by changing relief or
background color.
The radiobutton shall (when the indicator is turned on) display an dot
mark or other distinguishing icon/symbol that clearly indicates a
selected or On state. The unselected or Off state shall be displayed
with an empty diamond. The state values will ''not'' be indicated by
changing relief or background color.
~ Copyright
This document has been placed in the public domain.

# TIP 109: New Look for Checkbutton and Radiobutton on Unix
State: Final
Type: Project
Tcl-Version: 8.5
Vote: Done
Post-History:
Author: Brian Griffin <bgriffin@model.com>
Created: 01-Oct-2002
-----# Abstract
This TIP proposes changing the look of the Tk checkbutton and
radiobutton widgets on Unix to more closely match the Windows
counterparts.
# Rationale
The current visual aspect of the Unix version checkbutton and
radiobutton has proved to be confusing to users. The distinction
between selected \(On\) and unselected \(Off\) states are not visually
different enough to clearly identify one from another. Indeed, in the
rare case where only one checkbutton is present, one cannot tell for
certain if the state is On or Off. With a check or dot mark icon
\(dependent on the type of widget\) displayed in the Windows version,
there is no question about the state of the widget.
# Proposed Changes
The checkbutton shall \(when the indicator is turned on\) display an
check-mark or other distinguishing icon/symbol that clearly indicates
an On state. The Off state will be displayed with an empty box. The
state values will _not_ be indicated by changing relief or
background color.
The radiobutton shall \(when the indicator is turned on\) display an dot
mark or other distinguishing icon/symbol that clearly indicates a
selected or On state. The unselected or Off state shall be displayed
with an empty diamond. The state values will _not_ be indicated by
changing relief or background color.
# Copyright
This document has been placed in the public domain.

TIP: 110Title: Add a Tristate Mode to the Checkbutton and Radiobutton
Version: $Revision: 1.12 $
Author: Brian Griffin <bgriffin@model.com>
State: Final
Type: Project
Vote: Done
Created: 01-Oct-2002
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes adding a third value (tristate) to the checkbutton
and radiobutton widgets and corresponding display.
~ Rationale
In order to meet more demanding requirements of todays complex
graphical user interfaces, it would help add to the functionality of
the basic checkbutton and radiobutton widgets. To support using
checkbuttons and radiobuttons along with multiple selection, it is
necessary for these buttons to be able to display a third state,
i.e., both On and Off. This indicates the situation where a property
has a particular value for some members of a selection set, but
not others.
~ Proposed Change
The change would add a third "tristate" value to current On Off values
of the checkbutton and radiobutton widgets. The widget would then
display the check/dot mark (as appropriate) along with a "grayed"
background, for example. When the checkbutton or radiobutton is
invoked (i.e. clicked on) it would behave exactly as it does currently,
setting the variable to the On value.
There is a concern that the Unix version of these widgets do not have
a sufficiently different appearance when in the tristate state
compared with the On and Off states. This issue is addressed in
[109].
~ Propsal Specifics
1. Identify a third, platform specific, presentation (e.g. check
with grayed background) to represent the tristate value,
1. Add the option "-tristatevalue" to specify the match value
only. The default value of this option will be "{}",
1. Change the behavior to display the indeterminate or tristate
presentation when the associated variable's value matches the
-tristatevalue, and
1. Add the option "-tristateimage" to specify an image to display
(in place of the image option) when the checkbutton or
radiobutton has the tristate value (as defined above.)
~ Copyright
This document has been placed in the public domain.

# TIP 110: Add a Tristate Mode to the Checkbutton and Radiobutton
Author: Brian Griffin <bgriffin@model.com>
State: Final
Type: Project
Vote: Done
Created: 01-Oct-2002
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes adding a third value \(tristate\) to the checkbutton
and radiobutton widgets and corresponding display.
# Rationale
In order to meet more demanding requirements of todays complex
graphical user interfaces, it would help add to the functionality of
the basic checkbutton and radiobutton widgets. To support using
checkbuttons and radiobuttons along with multiple selection, it is
necessary for these buttons to be able to display a third state,
i.e., both On and Off. This indicates the situation where a property
has a particular value for some members of a selection set, but
not others.
# Proposed Change
The change would add a third "tristate" value to current On Off values
of the checkbutton and radiobutton widgets. The widget would then
display the check/dot mark \(as appropriate\) along with a "grayed"
background, for example. When the checkbutton or radiobutton is
invoked \(i.e. clicked on\) it would behave exactly as it does currently,
setting the variable to the On value.
There is a concern that the Unix version of these widgets do not have
a sufficiently different appearance when in the tristate state
compared with the On and Off states. This issue is addressed in
[[109]](109.md).
# Propsal Specifics
1. Identify a third, platform specific, presentation \(e.g. check
with grayed background\) to represent the tristate value,
1. Add the option "-tristatevalue" to specify the match value
only. The default value of this option will be "\{\}",
1. Change the behavior to display the indeterminate or tristate
presentation when the associated variable's value matches the
-tristatevalue, and
1. Add the option "-tristateimage" to specify an image to display
\(in place of the image option\) when the checkbutton or
radiobutton has the tristate value \(as defined above.\)
# Copyright
This document has been placed in the public domain.

TIP: 111Title: Dictionary Values and Manipulators
Version: $Revision: 1.12 $
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Author: David S. Cargo <dcargo@marix.com>
State: Final
Type: Project
Vote: Done
Created: 05-Oct-2002
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes adding a standard value format (and supporting
commands) to Tcl that implements a value-to-value mapping, just as
Tcl's list values can be regarded as implementing a number-to-value
mapping.
~ Rationale
What is a dictionary? It is a translation from arbitrary values to
arbitrary values, often also known as an associative map. Many
computer languages, especially higher-level ones, have them as part of
the language or the standard library. It would be nice to have them
in Tcl too.
................................................................................
Now, I realise that Tcl already contains arrays which provide
dictionary functionality, but they are not quite the same thing.
Tcl's arrays are collections of variables indexable by name, and not
collections of values. This has some far-reaching implications; it is
possible to set traces on individual elements of the array, but it is
not possible to pass the array by value. However, one of the main
concerns is the sheer cost of arrays in terms of memory space; aside
from the hash table used as the core of the implementation (and the
representations of the keys and values, of course) there is a
substantial overhead for each array to support traces on the array as
a whole, plus a similar overhead ''per element'' that stems from the
fact that elements are variables in their own right. By contrast, a
dictionary value should be a lot more frugal.
~ Value Syntax and Semantics
Naturally, it is desirable for dictionary values to have
human-readable forms that are similar to those that currently exist.
I propose using ''key value key value ...'' form with list-style
quoting for keys and values that contain characters that are
significant to Tcl, which should be immediately familiar to users of
the '''array get''' and '''array set''' commands. No special
interpretation will be placed on the amount of whitespace separating
keys and values, just as with lists (indeed, any list with an even
number of elements can be regarded as a dictionary.) For example, the
following value represents a mapping from selected languages to a
possible program to invoke to compile them:
| C gcc C++ g++ FORTRAN f77 Java javac
Empty dictionaries are those that contain no mappings from keys to
values. Any representation of an empty list will also be a
representation of an empty dictionary. There is no upper bound on the
number of items that a dictionary may hold.
It should be specially noted that dictionary values have copy-on-write
semantics just like lists. This means that if I hand a dictionary
value into a procedure as an argument, and that procedure updates the
variable containing that value, the value as seen by the caller will
not have changed. This is in complete contrast with arrays which
cannot (currently) be passed by value other than through using '''array
get''' to convert the array to a list form and '''array set''' to convert
back again.
This specification does not state what order the keys and values are
listed in. That depends on the implementation.
~ Command Syntax and Semantics
I propose that all operations that work with dictionary values (where
not done through adaptations of existing commands) will go through the
'''dict''' command. The alternatives are "array" which is already in
use, "dictionary" which is rather long for what I believe will be a
fairly commonly used command, "alist" (association list) which is
probably too easy to confuse with existing commands, and "map" which
is probably better reserved for future use as something for applying
an operation to a list (or other collection of values), "hash" (which
is perhaps too common), and "table" (which is used for this type of
data structure in the Icon programming language).
Most subcommands operate on either a dictionary value (''exists'',
''for'', ''get'', ''info'', ''keys'', ''remove'', ''replace'',
''size'', and ''values''), or on a variable containing a dictionary
value (''append'', ''incr'', ''lappend'', ''set'', and ''unset'').
Proposed subcommands:
dict create: Make a dictionary.
> '''dict create''' ?''key1 value1 key2 value2 ...''?
> This will create a new dictionary from the given keys and values
and return it as the result. The command will take an even number
of arbitrary strings (or other objects, naturally) and will use the
first, third, fifth, etc. as keys and the second, fourth, sixth,
etc. as values. From the point of view of string representations,
this command will behave the same as the '''list''' command with an
even number of arguments. There is no restriction on the possible
representations of keys or values. It is legal to call this
command with no arguments at all, which creates an empty
dictionary.
dict get: Get value for given key.
> '''dict get''' ''dictionaryValue'' ?''key ...''?
> Given a dictionary value (first argument) and a key (second
argument), this will retrieve the value for that key. Where
several keys are supplied, the behaviour of the command shall be as
if the result of '''dict get''' ''dictVal key'' was passed as the first
argument to '''dict get'' with the remaining arguments as second
(and possibly subsequent) arguments. This facilitates lookups in
nested dictionaries. For example, the following two commands are
equivalent:
| dict get $dict foo bar spong
| dict get [dict get [dict get $dict foo] bar] spong
> If no keys are provided, dict would return a list containing pairs
of elements in a manner similar to '''array get'''. That is, the
first element of each pair would be the key and the second element
would be the value for that key.
> It is an error to attempt to retrieve a value for a key that is not
present in the dictionary.
dict replace: Create a new dictionary that is a copy of an old one
except with some values different or some extra
key/value pairs added.
> '''dict replace''' ''dictionaryValue'' ?''key value ...''?
> This is very much the analogue of '''lreplace''', taking a dictionary
value as its first argument and then a list of key/value pairs.
The result of the command is a new dictionary value that is a copy
of the supplied dictionary other than that whenever a key is one of
those supplied to this command, the returned dictionary will map
that key to the associated value. It is legal for this command to
be called with no key/value pairs, but illegal for this command to
be called with a key but no value.
dict remove: Create a new dictionary that is a copy of an old one
except without the key/value mappings whose keys are
listed.
> '''dict remove''' ''dictionaryValue'' ?''key key ...''?
> This operation does what '''dict replace''' can't do; removes keys
and values. The result of the command is a new dictionary value
that does not contain mappings for any of the keys listed; it is
not an error if either there are no keys listed, or if any of the
listed keys does not exist in the supplied dictionary.
dict set: Set value for given key in a dictionary in a variable.
> '''dict set''' ''dictionaryVar key'' ?''key ...''? ''value''
> This operation takes the name of a variable containing a dictionary
value and places an updated dictionary value in that variable
containing a mapping from the given key to the given value. In a
manner analogous to '''lset''', where multiple keys are present, they
do indexing into nested dictionaries.
dict unset: Remove association for given key in a dictionary in a
variable.
> '''dict unset''' ''dictionaryVar key'' ?''key ...''?
> This operation takes the name of a variable containing a dictionary
value and places an updated dictionary value in that variable that
does not contain a mapping for the given key. Where multiple keys
are present, this describes a path through nested dictionaries to
the mapping to remove. At least one key must be specified.
dict keys: List all keys (with optional criteria matching) in
dictionary.
> '''dict keys''' ''dictionaryValue'' ?''globPattern''?
> Return a list of all keys in the given dictionary value. If a
pattern is supplied, only those keys that match it (according to
the rules of '''string match''') will be returned. The returned keys
will be in an arbitrary implementation-specific order.
dict values: List all values (with optional criteria matching) in
the dictionary.
> '''dict values''' ''dictionaryValue'' ?''globPattern''?
> Return a list of all values in the given dictionary value. If a
pattern is supplied, only those values that match it (according to
the rules of '''string match''') will be returned. The returned keys
will be in an arbitrary implementation-specific order, though where
no pattern is supplied the ''i''th key returned by '''dict keys'''
will be the key for the ''i''th value returned by '''dict values'''
applied to the same dictionary value.
dict for: Iterate across all key/value mappings in the
dictionary.
> '''dict for''' {''keyVar valueVar''} ''dictionaryValue body''
> This takes three arguments, the first a pair of variable names (for
the key and value respectively of each mapping in the dictionary),
the second the dictionary value to iterate across, and the third a
script to be evaluated for each mapping with the key and value
variables set appropriately (in the manner of '''foreach'''.) The
result of the command is an empty string. If any evaluation of the
body generates a ''TCL_BREAK'' result, no further pairs from the
dictionary will be iterated over and the '''dict for''' command will
terminate successfully immediately. If any evaluation of the body
generates a ''TCL_CONTINUE'' result, this shall be treated exactly
like a normal ''TCL_OK'' result.
dict filter: Create a new dictionary from an old one containing just
a selection of key/value pairs.
> '''dict filter''' ''dictionaryValue'' '''key''' ''globPattern''
> '''dict filter''' ''dictionaryValue'' '''value''' ''globPattern''
> '''dict filter''' ''dictionaryValue'' '''script''' {''keyVar valueVar''} ''script''
> This takes a dictionary value and returns a new dictionary that
contains just those key/value pairs that match the specified rule.
Three rules are outlined above. The '''key''' rule only matches those
key/value pairs whose keys match the given glob-style pattern. The
'''value''' rule only matches those key/value pairs whose values match
the given glob-style pattern. The '''script''' rule tests for matching
by assigning the key to the ''keyVar'' and the value to the
''valueVar'', and then evaluating the given script which should
return a boolean value (with the key/value pair only being included
in the result of the '''dict filter''' when a true value is returned.)
dict append: Append a string to the value for a particular key in
the dictionary.
> '''dict append''' ''dictionaryVar key'' ?''string ...''?
> This appends the given string (or strings) to the value that the
given key maps to in the dictionary value contained in the given
variable, writing the resulting dictionary value back to that
variable. Non-existent keys are treated as if they map to an empty
string.
dict incr: Increment the value for a particular key in the
dictionary.
> '''dict incr''' ''dictionaryVar key'' ?''increment''?
> This adds the given increment value (an integer that defaults to 1
if not specified) to the value that the given key maps to in the
dictionary value contained in the given variable, writing the
resulting dictionary value back to that variable. Non-existent
keys are treated as if they map to 0. It is an error to increment
a value for an existing key if that value is not an integer.
dict lappend: Append an item to the list-value for a particular key
in the dictionary.
> '''dict lappend''' ''dictionaryVar key'' ?''item ...''?
> This appends the given items to the list value that the given key
maps to in the dictionary value contained in the given variable,
writing the resulting dictionary value back to that variable.
Non-existent keys are treated as if they map to an empty list, and
it is legal for there to be no items to append to the list. It is
an error for the value that the key maps to to not be representable
as a list.
dict exists: Test whether a mapping exists for a key.
> '''dict exists''' ''dictionaryValue key'' ?''key ...''?
> This returns a boolean value indicating whether the given key (or
path of keys through a set of nested dictionaries) exists in the
given dictionary value. This returns a true value exactly when
'''dict get''' on that path will succeed.
dict size: Get the number of key/value mappings in a dictionary.
> '''dict size''' ''dictionaryValue''
> This returns the size of the dictionary, which will be exactly half
the value that '''llength''' ''dictionaryValue'' would return. It is an
error to apply this command to a non-dictionary value.
dict info: Get implementation-specific information about the
dictionary value.
> '''dict info''' ''dictionaryValue''
> This returns information (intended for display to people) about the
given dictionary though the format of this data is dependent on the
implementation of the dictionary. For dictionaries that are
implemented by hash tables, it is expected that this will return
the string produced by ''Tcl_HashStats()''.
~ Other Related Changes
There are a few other commands that change:
* '''array set''' will take a dictionary instead of (or as well as) a
list as its final argument.
* '''array get''' will return a dictionary.
* '''string map''' will take a dictionary instead of (or as well as) a
list as its map argument.
Naturally, dictionary handling will form its own maintenance area. [16] and [24] will be updated as necessary.
~ C API
There will be a new public structure and a few new public functions to
allow C-level access to dictionary values:
The new structure (called ''Tcl_DictSearch'') will be there to allow
for searches (i.e. traversals) of a dictionary. This TIP does not
specify the fields of the structure; the declaration is just to allow
for allocation of these structures on the C stack.
Many public API functions are capable of generating error messages;
these generally indicate some type-conversion failure. Sharing
constraint violations (where applicable) cause panics as they indicate
basic programming errors which should not be causable by scripts. The
public API functions are:
> Tcl_Obj *'''Tcl_NewDictObj'''(void);
> Tcl_Obj *'''Tcl_DbNewDictObj'''(CONST char *''file'', int ''line'');
These functions (in non-debug and debug versions) create a new
dictionary object and return it.
> int '''Tcl_DictObjPut'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
Tcl_Obj *''keyPtr'', Tcl_Obj *''valuePtr'');
This function inserts a new key/value pair into a dictionary, or
updates a key/value pair already in the dictionary. The dictionary
object must be unshared. Note that both the key and value objects
will have their reference count increased. The return value is a
normal TCL_OK/TCL_ERROR result, with the ''interp'' for error
reporting.
> int '''Tcl_DictObjGet'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
Tcl_Obj *''keyPtr'', Tcl_Obj **''valuePtrPtr'');
This function looks up the value for a key in a dictionary. The
variable pointed to by the ''valuePtrPtr'' argument is updated to
contain a reference to the value, or a NULL if there is no mapping for
the key in the dictionary. No reference counts are manipulated by
this function. The return value is a normal TCL_OK/TCL_ERROR result,
with the ''interp'' for error reporting.
> int '''Tcl_DictObjRemove'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
Tcl_Obj *''keyPtr'');
This function removes the key/value pair with the given key from the
dictionary. It is not an error if the key is not present in the
dictionary. The dictionary must be unshared. The return value is a
normal TCL_OK/TCL_ERROR result, with the ''interp'' for error
reporting.
> int '''Tcl_DictObjSize'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
int *''sizePtr'');
This function updates the integer variable pointed to by ''sizePtr''
with the number of key/value pairs in the dictionary. The return
value is a normal TCL_OK/TCL_ERROR result, with the ''interp'' for
error reporting.
> int '''Tcl_DictObjFirst'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
Tcl_DictSearch *''searchPtr'',
Tcl_Obj **''keyPtrPtr'', Tcl_Obj **''valuePtrPtr'', int *''donePtr'');
This function starts a search of (i.e. iteration over) the given
dictionary, using the structure pointed to by ''searchPtr'' as
context. The return value is a normal TCL_OK/TCL_ERROR result, with
the ''interp'' for error reporting. Three variables are updated to
indicate what was found; ''keyPtrPtr'' is used for reporting the key
of a key/value pair and ''valuePtrPtr'' is used for reporting the
corresponding value. Finally, ''donePtr'' is used for indicating
whether the search has found all the values; if the variable it points
to is set to 1 there are no key/value pairs in the dictionary
(i.e. the variables pointed to by ''keyPtrPtr'' and ''valuePtrPtr''
were not updated), but if the variable is set to 0, a key/value pair
was found and ''Tcl_DictObjNext()'' should be called to discover
whether that was the last value or if there are further ones in the
dictionary. Note that if this function indicates that the search is
not done but the calling code wishes to extract no further values from
the dictionary, ''Tcl_DictObjDone()'' ''must'' be called to release
the internal locks on the representation of the value.
> void '''Tcl_DictObjNext'''(Tcl_DictSearch *''searchPtr'',
Tcl_Obj **''keyPtrPtr'', Tcl_Obj **''valuePtrPtr'', int *''donePtr'');
This function gets the next key/value pair from the search of a
dictionary, using the search referenced by ''searchPtr''. The meaning
of the ''keyPtrPtr'', ''valuePtrPtr'' and ''donePtr'' variables is
much the same as in ''Tcl_DictObjFirst()'', along with the restriction
that if the function indicates that the search is not done but the
calling code wishes to extract no further values from the dictionary,
''Tcl_DictObjDone()'' ''must'' be called to release the internal locks
on the representation of the value.
> void '''Tcl_DictObjDone'''(Tcl_DictSearch *''searchPtr'');
This function terminates a search of a dictionary before all the
values in the dictionary have been iterated over, releasing the
internal locks on the dictionary representation.
> int '''Tcl_DictObjPutKeyList'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
int ''keyc'', Tcl_Obj *CONST *''keyv'', Tcl_Obj *''valuePtr'');
This function is a variant on ''Tcl_DictObjPut()'' that takes a list
of keys so as to work with nested dictionaries.
> int '''Tcl_DictObjRemoveKeyList'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
int ''keyc'', Tcl_Obj *CONST *''keyv'');
This function is a variant on ''Tcl_DictObjRemove()'' that takes a
list of keys so as to work with nested dictionaries.
~ Examples
Counting the number of unique words in a file and the number of times
each word occurs:
|set f [open someFile.txt]
|set contents [read $f]
|close $f
|foreach word [regexp -all -inline {\w+} $contents] {
| dict incr count $word
|}|puts "There are [dict size $count] unique words."
|foreach word [lsort -dictionary [dict keys $count]] {
| puts "${word}: [dict get $count $word] occurrences"
|}
A localisable '''string toupper''' implementation:
|set capital [dict create C [dict create]]
|foreach c {abcdefghijklmnopqrstuvwxyz} {
| dict set capital C $c [string toupper $c]
|}|dict set capital en [dict get $capital C]
|# ... and so on for other supported languages ...
|set upperCase [string map [dict get $capital $env(LANG)] $string
~ Copyright
This document has been placed in the public domain.
----
~ Appendices
''These appendices are not formally part of the proposal and exist
merely to help understanding.''~ ~ Implementation Notes
Implement using hash tables (of course.) Need efficient ways to
convert to/from lists, perhaps making lists know what's going on
underneath the covers?
~ ~ Future Directions
Alternate implementations of mappings, like trees or disk-backed
databases?

# TIP 111: Dictionary Values and Manipulators
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Author: David S. Cargo <dcargo@marix.com>
State: Final
Type: Project
Vote: Done
Created: 05-Oct-2002
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes adding a standard value format \(and supporting
commands\) to Tcl that implements a value-to-value mapping, just as
Tcl's list values can be regarded as implementing a number-to-value
mapping.
# Rationale
What is a dictionary? It is a translation from arbitrary values to
arbitrary values, often also known as an associative map. Many
computer languages, especially higher-level ones, have them as part of
the language or the standard library. It would be nice to have them
in Tcl too.
................................................................................
Now, I realise that Tcl already contains arrays which provide
dictionary functionality, but they are not quite the same thing.
Tcl's arrays are collections of variables indexable by name, and not
collections of values. This has some far-reaching implications; it is
possible to set traces on individual elements of the array, but it is
not possible to pass the array by value. However, one of the main
concerns is the sheer cost of arrays in terms of memory space; aside
from the hash table used as the core of the implementation \(and the
representations of the keys and values, of course\) there is a
substantial overhead for each array to support traces on the array as
a whole, plus a similar overhead _per element_ that stems from the
fact that elements are variables in their own right. By contrast, a
dictionary value should be a lot more frugal.
# Value Syntax and Semantics
Naturally, it is desirable for dictionary values to have
human-readable forms that are similar to those that currently exist.
I propose using _key value key value ..._ form with list-style
quoting for keys and values that contain characters that are
significant to Tcl, which should be immediately familiar to users of
the **array get** and **array set** commands. No special
interpretation will be placed on the amount of whitespace separating
keys and values, just as with lists \(indeed, any list with an even
number of elements can be regarded as a dictionary.\) For example, the
following value represents a mapping from selected languages to a
possible program to invoke to compile them:
C gcc C++ g++ FORTRAN f77 Java javac
Empty dictionaries are those that contain no mappings from keys to
values. Any representation of an empty list will also be a
representation of an empty dictionary. There is no upper bound on the
number of items that a dictionary may hold.
It should be specially noted that dictionary values have copy-on-write
semantics just like lists. This means that if I hand a dictionary
value into a procedure as an argument, and that procedure updates the
variable containing that value, the value as seen by the caller will
not have changed. This is in complete contrast with arrays which
cannot \(currently\) be passed by value other than through using **array
get** to convert the array to a list form and **array set** to convert
back again.
This specification does not state what order the keys and values are
listed in. That depends on the implementation.
# Command Syntax and Semantics
I propose that all operations that work with dictionary values \(where
not done through adaptations of existing commands\) will go through the
**dict** command. The alternatives are "array" which is already in
use, "dictionary" which is rather long for what I believe will be a
fairly commonly used command, "alist" \(association list\) which is
probably too easy to confuse with existing commands, and "map" which
is probably better reserved for future use as something for applying
an operation to a list \(or other collection of values\), "hash" \(which
is perhaps too common\), and "table" \(which is used for this type of
data structure in the Icon programming language\).
Most subcommands operate on either a dictionary value \(_exists_,
_for_, _get_, _info_, _keys_, _remove_, _replace_,
_size_, and _values_\), or on a variable containing a dictionary
value \(_append_, _incr_, _lappend_, _set_, and _unset_\).
Proposed subcommands:
dict create: Make a dictionary.
> **dict create** ?_key1 value1 key2 value2 ..._?
> This will create a new dictionary from the given keys and values
and return it as the result. The command will take an even number
of arbitrary strings \(or other objects, naturally\) and will use the
first, third, fifth, etc. as keys and the second, fourth, sixth,
etc. as values. From the point of view of string representations,
this command will behave the same as the **list** command with an
even number of arguments. There is no restriction on the possible
representations of keys or values. It is legal to call this
command with no arguments at all, which creates an empty
dictionary.
dict get: Get value for given key.
> **dict get** _dictionaryValue_ ?_key ..._?
> Given a dictionary value \(first argument\) and a key \(second
argument\), this will retrieve the value for that key. Where
several keys are supplied, the behaviour of the command shall be as
if the result of **dict get** _dictVal key_ was passed as the first
argument to **dict get_ with the remaining arguments as second
\(and possibly subsequent\) arguments. This facilitates lookups in
nested dictionaries. For example, the following two commands are
equivalent:
dict get $dict foo bar spong
dict get [dict get [dict get $dict foo] bar] spong
> If no keys are provided, dict would return a list containing pairs
of elements in a manner similar to **array get**. That is, the
first element of each pair would be the key and the second element
would be the value for that key.
> It is an error to attempt to retrieve a value for a key that is not
present in the dictionary.
dict replace: Create a new dictionary that is a copy of an old one
except with some values different or some extra
key/value pairs added.
> **dict replace** _dictionaryValue_ ?_key value ..._?
> This is very much the analogue of **lreplace**, taking a dictionary
value as its first argument and then a list of key/value pairs.
The result of the command is a new dictionary value that is a copy
of the supplied dictionary other than that whenever a key is one of
those supplied to this command, the returned dictionary will map
that key to the associated value. It is legal for this command to
be called with no key/value pairs, but illegal for this command to
be called with a key but no value.
dict remove: Create a new dictionary that is a copy of an old one
except without the key/value mappings whose keys are
listed.
> **dict remove** _dictionaryValue_ ?_key key ..._?
> This operation does what **dict replace** can't do; removes keys
and values. The result of the command is a new dictionary value
that does not contain mappings for any of the keys listed; it is
not an error if either there are no keys listed, or if any of the
listed keys does not exist in the supplied dictionary.
dict set: Set value for given key in a dictionary in a variable.
> **dict set** _dictionaryVar key_ ?_key ..._? _value_
> This operation takes the name of a variable containing a dictionary
value and places an updated dictionary value in that variable
containing a mapping from the given key to the given value. In a
manner analogous to **lset**, where multiple keys are present, they
do indexing into nested dictionaries.
dict unset: Remove association for given key in a dictionary in a
variable.
> **dict unset** _dictionaryVar key_ ?_key ..._?
> This operation takes the name of a variable containing a dictionary
value and places an updated dictionary value in that variable that
does not contain a mapping for the given key. Where multiple keys
are present, this describes a path through nested dictionaries to
the mapping to remove. At least one key must be specified.
dict keys: List all keys \(with optional criteria matching\) in
dictionary.
> **dict keys** _dictionaryValue_ ?_globPattern_?
> Return a list of all keys in the given dictionary value. If a
pattern is supplied, only those keys that match it \(according to
the rules of **string match**\) will be returned. The returned keys
will be in an arbitrary implementation-specific order.
dict values: List all values \(with optional criteria matching\) in
the dictionary.
> **dict values** _dictionaryValue_ ?_globPattern_?
> Return a list of all values in the given dictionary value. If a
pattern is supplied, only those values that match it \(according to
the rules of **string match**\) will be returned. The returned keys
will be in an arbitrary implementation-specific order, though where
no pattern is supplied the _i_th key returned by **dict keys**
will be the key for the _i_th value returned by **dict values**
applied to the same dictionary value.
dict for: Iterate across all key/value mappings in the
dictionary.
> **dict for** \{_keyVar valueVar_\} _dictionaryValue body_
> This takes three arguments, the first a pair of variable names \(for
the key and value respectively of each mapping in the dictionary\),
the second the dictionary value to iterate across, and the third a
script to be evaluated for each mapping with the key and value
variables set appropriately \(in the manner of **foreach**.\) The
result of the command is an empty string. If any evaluation of the
body generates a _TCL\_BREAK_ result, no further pairs from the
dictionary will be iterated over and the **dict for** command will
terminate successfully immediately. If any evaluation of the body
generates a _TCL\_CONTINUE_ result, this shall be treated exactly
like a normal _TCL\_OK_ result.
dict filter: Create a new dictionary from an old one containing just
a selection of key/value pairs.
> **dict filter** _dictionaryValue_ **key** _globPattern_
> **dict filter** _dictionaryValue_ **value** _globPattern_
> **dict filter** _dictionaryValue_ **script** \{_keyVar valueVar_\} _script_
> This takes a dictionary value and returns a new dictionary that
contains just those key/value pairs that match the specified rule.
Three rules are outlined above. The **key** rule only matches those
key/value pairs whose keys match the given glob-style pattern. The
**value** rule only matches those key/value pairs whose values match
the given glob-style pattern. The **script** rule tests for matching
by assigning the key to the _keyVar_ and the value to the
_valueVar_, and then evaluating the given script which should
return a boolean value \(with the key/value pair only being included
in the result of the **dict filter** when a true value is returned.\)
dict append: Append a string to the value for a particular key in
the dictionary.
> **dict append** _dictionaryVar key_ ?_string ..._?
> This appends the given string \(or strings\) to the value that the
given key maps to in the dictionary value contained in the given
variable, writing the resulting dictionary value back to that
variable. Non-existent keys are treated as if they map to an empty
string.
dict incr: Increment the value for a particular key in the
dictionary.
> **dict incr** _dictionaryVar key_ ?_increment_?
> This adds the given increment value \(an integer that defaults to 1
if not specified\) to the value that the given key maps to in the
dictionary value contained in the given variable, writing the
resulting dictionary value back to that variable. Non-existent
keys are treated as if they map to 0. It is an error to increment
a value for an existing key if that value is not an integer.
dict lappend: Append an item to the list-value for a particular key
in the dictionary.
> **dict lappend** _dictionaryVar key_ ?_item ..._?
> This appends the given items to the list value that the given key
maps to in the dictionary value contained in the given variable,
writing the resulting dictionary value back to that variable.
Non-existent keys are treated as if they map to an empty list, and
it is legal for there to be no items to append to the list. It is
an error for the value that the key maps to to not be representable
as a list.
dict exists: Test whether a mapping exists for a key.
> **dict exists** _dictionaryValue key_ ?_key ..._?
> This returns a boolean value indicating whether the given key \(or
path of keys through a set of nested dictionaries\) exists in the
given dictionary value. This returns a true value exactly when
**dict get** on that path will succeed.
dict size: Get the number of key/value mappings in a dictionary.
> **dict size** _dictionaryValue_
> This returns the size of the dictionary, which will be exactly half
the value that **llength** _dictionaryValue_ would return. It is an
error to apply this command to a non-dictionary value.
dict info: Get implementation-specific information about the
dictionary value.
> **dict info** _dictionaryValue_
> This returns information \(intended for display to people\) about the
given dictionary though the format of this data is dependent on the
implementation of the dictionary. For dictionaries that are
implemented by hash tables, it is expected that this will return
the string produced by _Tcl\_HashStats\(\)_.
# Other Related Changes
There are a few other commands that change:
* **array set** will take a dictionary instead of \(or as well as\) a
list as its final argument.
* **array get** will return a dictionary.
* **string map** will take a dictionary instead of \(or as well as\) a
list as its map argument.
Naturally, dictionary handling will form its own maintenance area. [[16]](16.md) and [[24]](24.md) will be updated as necessary.
# C API
There will be a new public structure and a few new public functions to
allow C-level access to dictionary values:
The new structure \(called _Tcl\_DictSearch_\) will be there to allow
for searches \(i.e. traversals\) of a dictionary. This TIP does not
specify the fields of the structure; the declaration is just to allow
for allocation of these structures on the C stack.
Many public API functions are capable of generating error messages;
these generally indicate some type-conversion failure. Sharing
constraint violations \(where applicable\) cause panics as they indicate
basic programming errors which should not be causable by scripts. The
public API functions are:
> Tcl\_Obj \***Tcl\_NewDictObj**\(void\);
> Tcl\_Obj \***Tcl\_DbNewDictObj**\(CONST char \*_file_, int _line_\);
These functions \(in non-debug and debug versions\) create a new
dictionary object and return it.
> int **Tcl\_DictObjPut**\(Tcl\_Interp \*_interp_, Tcl\_Obj \*_dictPtr_,
Tcl\_Obj \*_keyPtr_, Tcl\_Obj \*_valuePtr_\);
This function inserts a new key/value pair into a dictionary, or
updates a key/value pair already in the dictionary. The dictionary
object must be unshared. Note that both the key and value objects
will have their reference count increased. The return value is a
normal TCL\_OK/TCL\_ERROR result, with the _interp_ for error
reporting.
> int **Tcl\_DictObjGet**\(Tcl\_Interp \*_interp_, Tcl\_Obj \*_dictPtr_,
Tcl\_Obj \*_keyPtr_, Tcl\_Obj \*\*_valuePtrPtr_\);
This function looks up the value for a key in a dictionary. The
variable pointed to by the _valuePtrPtr_ argument is updated to
contain a reference to the value, or a NULL if there is no mapping for
the key in the dictionary. No reference counts are manipulated by
this function. The return value is a normal TCL\_OK/TCL\_ERROR result,
with the _interp_ for error reporting.
> int **Tcl\_DictObjRemove**\(Tcl\_Interp \*_interp_, Tcl\_Obj \*_dictPtr_,
Tcl\_Obj \*_keyPtr_\);
This function removes the key/value pair with the given key from the
dictionary. It is not an error if the key is not present in the
dictionary. The dictionary must be unshared. The return value is a
normal TCL\_OK/TCL\_ERROR result, with the _interp_ for error
reporting.
> int **Tcl\_DictObjSize**\(Tcl\_Interp \*_interp_, Tcl\_Obj \*_dictPtr_,
int \*_sizePtr_\);
This function updates the integer variable pointed to by _sizePtr_
with the number of key/value pairs in the dictionary. The return
value is a normal TCL\_OK/TCL\_ERROR result, with the _interp_ for
error reporting.
> int **Tcl\_DictObjFirst**\(Tcl\_Interp \*_interp_, Tcl\_Obj \*_dictPtr_,
Tcl\_DictSearch \*_searchPtr_,
Tcl\_Obj \*\*_keyPtrPtr_, Tcl\_Obj \*\*_valuePtrPtr_, int \*_donePtr_\);
This function starts a search of \(i.e. iteration over\) the given
dictionary, using the structure pointed to by _searchPtr_ as
context. The return value is a normal TCL\_OK/TCL\_ERROR result, with
the _interp_ for error reporting. Three variables are updated to
indicate what was found; _keyPtrPtr_ is used for reporting the key
of a key/value pair and _valuePtrPtr_ is used for reporting the
corresponding value. Finally, _donePtr_ is used for indicating
whether the search has found all the values; if the variable it points
to is set to 1 there are no key/value pairs in the dictionary
\(i.e. the variables pointed to by _keyPtrPtr_ and _valuePtrPtr_
were not updated\), but if the variable is set to 0, a key/value pair
was found and _Tcl\_DictObjNext\(\)_ should be called to discover
whether that was the last value or if there are further ones in the
dictionary. Note that if this function indicates that the search is
not done but the calling code wishes to extract no further values from
the dictionary, _Tcl\_DictObjDone\(\)_ _must_ be called to release
the internal locks on the representation of the value.
> void **Tcl\_DictObjNext**\(Tcl\_DictSearch \*_searchPtr_,
Tcl\_Obj \*\*_keyPtrPtr_, Tcl\_Obj \*\*_valuePtrPtr_, int \*_donePtr_\);
This function gets the next key/value pair from the search of a
dictionary, using the search referenced by _searchPtr_. The meaning
of the _keyPtrPtr_, _valuePtrPtr_ and _donePtr_ variables is
much the same as in _Tcl\_DictObjFirst\(\)_, along with the restriction
that if the function indicates that the search is not done but the
calling code wishes to extract no further values from the dictionary,
_Tcl\_DictObjDone\(\)_ _must_ be called to release the internal locks
on the representation of the value.
> void **Tcl\_DictObjDone**\(Tcl\_DictSearch \*_searchPtr_\);
This function terminates a search of a dictionary before all the
values in the dictionary have been iterated over, releasing the
internal locks on the dictionary representation.
> int **Tcl\_DictObjPutKeyList**\(Tcl\_Interp \*_interp_, Tcl\_Obj \*_dictPtr_,
int _keyc_, Tcl\_Obj \*CONST \*_keyv_, Tcl\_Obj \*_valuePtr_\);
This function is a variant on _Tcl\_DictObjPut\(\)_ that takes a list
of keys so as to work with nested dictionaries.
> int **Tcl\_DictObjRemoveKeyList**\(Tcl\_Interp \*_interp_, Tcl\_Obj \*_dictPtr_,
int _keyc_, Tcl\_Obj \*CONST \*_keyv_\);
This function is a variant on _Tcl\_DictObjRemove\(\)_ that takes a
list of keys so as to work with nested dictionaries.
# Examples
Counting the number of unique words in a file and the number of times
each word occurs:
set f [open someFile.txt]
set contents [read $f]
close $f
foreach word [regexp -all -inline {\w+} $contents] {
dict incr count $word
}puts "There are [dict size $count] unique words."
foreach word [lsort -dictionary [dict keys $count]] {
puts "${word}: [dict get $count $word] occurrences"
}
A localisable **string toupper** implementation:
set capital [dict create C [dict create]]
foreach c {abcdefghijklmnopqrstuvwxyz} {
dict set capital C $c [string toupper $c]
}dict set capital en [dict get $capital C]
# ... and so on for other supported languages ...
set upperCase [string map [dict get $capital $env(LANG)] $string
# Copyright
This document has been placed in the public domain.
----
# Appendices
_These appendices are not formally part of the proposal and exist
merely to help understanding._# ~ Implementation Notes
Implement using hash tables \(of course.\) Need efficient ways to
convert to/from lists, perhaps making lists know what's going on
underneath the covers?
# ~ Future Directions
Alternate implementations of mappings, like trees or disk-backed
databases?

TIP: 112Title: Ensembles are Namespaces are Commands
Version: $Revision: 2.28 $
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 10-Oct-2002
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes unifying the concept of ensembles (from [[Incr
Tcl]]) with namespaces and commands. It also adds control of command
rewriting to allow for more efficient support for object systems like
Snit.
~ Rationale
Tcl's subcommand-style command collections (e.g. ''array'', ''info'',
''string'', ''interp'', etc.) are a very intuitive and popular way of
structuring collections of related commands. However, it is quite
awkward to write Tcl code that behaves that way. Users of [[Incr
Tcl]] have access to ensembles which provide that, but it would be a
very useful feature for many other uses too.
At the same time, it is becoming clear that many applications want to
commonly refer to commands inside other namespaces directly (instead
of through the [[namespace import]] mechanism) but the syntax for
doing this is verbose and not as elegant as it might be.
I believe that the same solution can address these two problems in one
go, and make the language stronger and more usable for it.
Furthermore, by giving the programmer control over the mapping from
the ensemble subcommands to their implementing commands, we can build
a simple class system on the cheap since we can, in effect, import
commands from elsewhere into the ensemble. By extending the mapping
so that it allows the specification of not just the implementing
command, but also some leading arguments to that command (similar to
what you can do with the [[interp alias]] mechanism) this becomes a
very powerful mechanism indeed.
Finally, a sophisticated extra capability is the addition of an unknown
subcommand callback to allow the creator of the ensemble to specify a
customized strategy for handling subcommands that are not recognized by
the ensemble machinery. This allows an ensemble to be dynamically
updated to include those subcommands that the user asks for, which is
a potential route for doing things like using an ensemble as a wrapper
round a Tk widget.
~ Proposed Change
I propose to add a new subcommand to the [[namespace]] command,
''ensemble'', that creates and manipulates the ensemble command for a
namespace. Each namespace may have any number of ensemble commands
associated with it, with the default name of an ensemble command
being the fully-qualified name of the namespace itself, though it will
be legal to rename ensemble commands (anyone wanting to track such
events should use the [[trace]] command.) Tcl will not create an
ensemble command for any namespace by default.
The [[namespace ensemble]] command will have the following subcommands:
create: For creating a new ensemble for the ''current'' namespace. The
command takes a list of option value pairs (as defined below) to
set the ensemble up with. As stated above, the default command name
is exactly the name of the namespace, but any other name may be
supplied (via the -command option); if it does not start with a
namespace separator, the name will be taken as being relative to
the current namespace. If another command with the same name exists,
it is deleted as part of the creation of the ensemble.
configure: For reading and writing the configuration of a particular
ensemble. The command takes an argument specifying the name of the
ensemble to configure, and then works with either no extra arguments
(when it retrieves the entire ensemble configuration), one extra
argument (the name of the option to retrieve), or a list of option
value pairs to configure the ensemble with. This command returns an error if applied to anything that is not an ensemble.
exists: This subcommand (which takes a single argument) tests whether
a command (with the given name) exists and is an ensemble. This command only returns an error if the wrong number of arguments are supplied.
The options available for creation and configuring are:
-subcommands: This option (if non-empty) specifies a list of ensemble
subcommands that the ensemble supports. It does not need to be
sorted. Each command is mapped according to the dictionary in the
''-map'' option (if a non-empty map is present and the command has
a mapping) or to the correspondingly named command (which is not
required to exist at the time this is specified) in the context
namespace.
-map: This option (if non-empty) specifies a dictionary that maps
from ensemble subcommand names to lists of arguments to substitute
into the ensemble invokation in place of the ensemble command name
and the subcommand name. See the ''-subcommands'' option for the meaning
of this option when that option is also non-empty. If this option is
empty and the ''-subcommands'' option is empty too, the namespace will use
the exported commands of the namespace as its command set, dynamically
determining them (subject to cacheing) every time the ensemble is
''invoked''. Note that the words in the dictionary values that give
the commands to map to (as opposed to any arguments to them) are
always resolved (if not absolute) relative to the namespace which is
running [[namespace ensemble create]] or [[namespace ensemble
configure]].
-prefixes: This boolean option (which is on by default) controls whether
unambiguous prefixes of ensemble commands are recognized as if they
were the fully specified ensemble command names.
-unknown: This provides (when non-empty) a partial command to handle
the case where an ensemble subcommand is not recognized and would
otherwise generate an error. When empty (the default) an error (in
the style of ''Tcl_GetIndexFromObj'') is generated whenever the
ensemble is unable to determine how to implement a particular
subcommand.
> See ''Unknown Handler Behaviour'' below for details of how the
ensemble interacts with its unknown handler.
Ensemble creation takes an extra option value pair:
-command: This option allows you to override what the name of the
ensemble to create is.
Ensemble configuring allows the querying of an extra read-only option:
-namespace: This ''read-only'' option allows the retrieval of the name
of the namespace that the ensemble was created in.
Given an ensemble command created by the above mechanism, calling the
command will first of all match the subcommand to its implementing
command (or command/argument list, as derived from the dictionary) in
a manner that will be recognizably similar to that enforced by
''Tcl_GetIndexFromObj()'' (unless the ''-unknown'' option override this
behaviour.) For details of how the ensemble determines what its subcommands are, please see the ''-subcommands'' and ''-map'' options above.
Then the ensemble command will rewrite the command and arguments so
that the ensemble command and subcommand are replaced by the
implementing command and any specified arguments, with the resulting
word list being fed to ''Tcl_EvalObjv()'' for execution. Note that
this does not increase the stack depth in terms of [[uplevel]], and
that the implementing command may itself be an ensemble command.
Note that if the namespace for an ensemble is deleted, the ensemble will
also be deleted though there is no lifetime constraint in the other
direction (i.e. using [[rename $theEnsemble {}]] will not cause the
namespace to vanish.) Obviously, any ensemble which has the global
namespace backing it up will have the same natural lifespan as the
hosting Tcl interpreter.
~ Unknown Handler Behaviour
If an unknown handler is specified for an ensemble, that handler is called when the ensemble command would otherwise return an error due to it being unable to decide which subcommand to invoke. The exact conditions under which that occurs are controlled by the ''-subcommands'', ''-map'' and ''-prefixes'' options as described above.
To execute the unknown handler, the ensemble mechanism takes the specified -unknown option and appends each argument of the attempted ensemble command invocation (including the ensemble command itself, expressed as a fully qualified name). It invokes the resulting command in the scope of the attempted call. If the execution of the unknown handler terminates normally, the ensemble engine reparses the subcommand (as described below) and tries to dispatch it again, which is ideal for when the ensemble's configuration has been updated by the unknown subcommand handler. Any other kind of termination of the unknown handler is treated as an error.
The result of the unknown handler is expected to be a list (it is an error if it is not). If the list is an empty list, the ensemble command attempts to look up the original subcommand again and, if it is not found this time, an error will be generated just as if the -unknown handler was not there (i.e. for any particular invokation of an ensemble, its unknown handler will be called at most once.) This makes it easy for the unknown handler to update the ensemble or its backing namespace so as to provide an implementation of the desired subcommand and reparse.
When the result is a non-empty list, the words of that list are used to replace the ensemble command and subcommand, just as if they had been looked up in the -map. It is up to the unknown handler to supply all namespace qualifiers if the implementing subcommand is not in the namespace of the caller of the ensemble command. Also note that when ensemble commands are chained (e.g. if you make one of the commands that implement an ensemble subcommand into an ensemble, in a manner similar to the text widget's ''tag'' and ''mark'' subcommands) then the rewrite happens in the context of the caller of the outermost ensemble. That is to say that ensembles do not in themselves place any namespace contexts on the Tcl call stack.
Where an empty -unknown handler is given (the default), the ensemble command will generate an error message based on the list of commands that the ensemble has defined (formatted similarly to the error message from ''Tcl_GetIndexFromObj()''). This is the error that will be thrown when the subcommand is still not recognized during reparsing. It is also an error for an -unknown handler to delete its namespace.
~ Examples
|namespace eval carrot {
| namespace export foo bar potato
| namespace ensemble create ;# Creates command ::carrot
|| proc foo {} {puts 1} ;# Exported
| proc bar {} {puts 2} ;# Exported
| proc boo {} {puts 3} ;# Not exported
|| namespace eval turnip { ;# Not exported
| namespace export alpha
| proc alpha {} {puts 4} ;# Exported
| proc beta {} {puts 5} ;# Not exported
| namespace ensemble create
| }|| namespace eval potato { ;# Exported
| proc north {x} {puts 6,$x} ;# Not exported
| proc south {x} {puts 7,$x} ;# Not exported
| # Notice we resolve names locally, and Tcl metachars are not special
| namespace ensemble create -map {
| north {north [$meta[$chars}
| }| }|}|||carrot foo ;# Prints 1
|carrot bar ;# Prints 2
|carrot b ;# Also prints 2 ("boo" not exported)
|carrot ? ;# ERROR: Alternatives "bar", "foo" and "potato"
|carrot potato ;# ERROR: Missing argument
|carrot potato ? ;# ERROR: Try "north" instead
|carrot potato north ;# Prints 6,[$meta[$chars
|carrot turnip alpha ;# ERROR: "turnip" not known
|carrot::turnip alpha ;# Prints 4
|carrot::turnip::beta ;# Prints 5
||rename ::carrot::potato ::spud
|spud north ;# Prints 6,[$meta[$chars
|spud south ;# ERROR: "south" not known
|carrot potato north ;# ERROR: No ::carrot::potato command
|# Reconfigure spud; notice we get different name resolution now!
|namespace ensemble configure spud -map {
| north {puts NORTH} south {puts SOUTH}
|}|spud north ;# Prints NORTH
|spud south ;# Prints SOUTH
|namespace delete carrot
|spud north ;# ERROR: spud command already deleted
|||namespace eval A {
| proc a args {puts A::a=>$args}
|}|namespace eval B {
| proc b args {puts B::b=>$args}
|}|# Create an ensemble in the global namespace
|namespace ensemble create -command C -map {
| eg1 {::A::a foo bar}
| eg2 {::B::b 1 2 3}
| eg3 ::string
|}|C eg1 spong ;# Prints A::a=>foo bar spong
|C eg2 evil code {[exit]} ;# Prints B::b=>1 2 3 evil code [exit]
|C eg3 length qwertyuiop ;# Returns 10
|||# An example demonstrating the use of -unknown to do delegation
|# This uses an ensemble to add a subcommand to a frame
|package require Tk
|rename [frame .f] _impl.f
|namespace ensemble create -command .f -unknown {delegate _impl.f} -map {
| flash {flashFrame _impl.f}
|}|# General delegation framework handler
|proc delegate {target ensemble subcmd args} {
| # Read the current map
| set map [namespace ensemble configure $ensemble -map]
| # Update it to include the new subcommand
| dict set map $subcmd [list $target $subcmd]
| # Install back into the ensemble
| namespace ensemble configure $ensemble -map $map
| # Result is empty string, so we reparse
| return {}
|}|# Our new subcommand implementation
|proc flashFrame w {
| set bg [$w cget -background]
| foreach colour {black white black white black white} {
| $w configure -background $colour
| update idletasks
| after 150
| }| $w configure -background $bg
|}~ Consequences
Many commands in both Tcl and Tk would benefit from leveraging this,
and it would enable straight-forward implementations of things like
[65] in pure Tcl code. It would also make doing things like partial
exposure of ensemble-like commands in safe interpreters much easier.
~ Sample Implementation
http://sf.net/tracker/?func=detail&aid=786509&group_id=10894&atid=310894~ Copyright
This document has been placed in the public domain.
~ Acknowledgements
Thanks very much to Joe English, Don Porter and Kevin Kenny for their
suggestions in the development of this TIP. Without them, it would have
been a far worse suggestion. And thanks to Will Duquette for writing a
piece of software (Snit) that would benefit immensely from pushing the
ensemble stuff as hard as possible and then a bit. :^)

# TIP 112: Ensembles are Namespaces are Commands
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 10-Oct-2002
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes unifying the concept of ensembles \(from [Incr
Tcl]\) with namespaces and commands. It also adds control of command
rewriting to allow for more efficient support for object systems like
Snit.
# Rationale
Tcl's subcommand-style command collections \(e.g. _array_, _info_,
_string_, _interp_, etc.\) are a very intuitive and popular way of
structuring collections of related commands. However, it is quite
awkward to write Tcl code that behaves that way. Users of [Incr
Tcl] have access to ensembles which provide that, but it would be a
very useful feature for many other uses too.
At the same time, it is becoming clear that many applications want to
commonly refer to commands inside other namespaces directly \(instead
of through the [namespace import] mechanism\) but the syntax for
doing this is verbose and not as elegant as it might be.
I believe that the same solution can address these two problems in one
go, and make the language stronger and more usable for it.
Furthermore, by giving the programmer control over the mapping from
the ensemble subcommands to their implementing commands, we can build
a simple class system on the cheap since we can, in effect, import
commands from elsewhere into the ensemble. By extending the mapping
so that it allows the specification of not just the implementing
command, but also some leading arguments to that command \(similar to
what you can do with the [interp alias] mechanism\) this becomes a
very powerful mechanism indeed.
Finally, a sophisticated extra capability is the addition of an unknown
subcommand callback to allow the creator of the ensemble to specify a
customized strategy for handling subcommands that are not recognized by
the ensemble machinery. This allows an ensemble to be dynamically
updated to include those subcommands that the user asks for, which is
a potential route for doing things like using an ensemble as a wrapper
round a Tk widget.
# Proposed Change
I propose to add a new subcommand to the [namespace] command,
_ensemble_, that creates and manipulates the ensemble command for a
namespace. Each namespace may have any number of ensemble commands
associated with it, with the default name of an ensemble command
being the fully-qualified name of the namespace itself, though it will
be legal to rename ensemble commands \(anyone wanting to track such
events should use the [trace] command.\) Tcl will not create an
ensemble command for any namespace by default.
The [namespace ensemble] command will have the following subcommands:
create: For creating a new ensemble for the _current_ namespace. The
command takes a list of option value pairs \(as defined below\) to
set the ensemble up with. As stated above, the default command name
is exactly the name of the namespace, but any other name may be
supplied \(via the -command option\); if it does not start with a
namespace separator, the name will be taken as being relative to
the current namespace. If another command with the same name exists,
it is deleted as part of the creation of the ensemble.
configure: For reading and writing the configuration of a particular
ensemble. The command takes an argument specifying the name of the
ensemble to configure, and then works with either no extra arguments
\(when it retrieves the entire ensemble configuration\), one extra
argument \(the name of the option to retrieve\), or a list of option
value pairs to configure the ensemble with. This command returns an error if applied to anything that is not an ensemble.
exists: This subcommand \(which takes a single argument\) tests whether
a command \(with the given name\) exists and is an ensemble. This command only returns an error if the wrong number of arguments are supplied.
The options available for creation and configuring are:
-subcommands: This option \(if non-empty\) specifies a list of ensemble
subcommands that the ensemble supports. It does not need to be
sorted. Each command is mapped according to the dictionary in the
_-map_ option \(if a non-empty map is present and the command has
a mapping\) or to the correspondingly named command \(which is not
required to exist at the time this is specified\) in the context
namespace.
-map: This option \(if non-empty\) specifies a dictionary that maps
from ensemble subcommand names to lists of arguments to substitute
into the ensemble invokation in place of the ensemble command name
and the subcommand name. See the _-subcommands_ option for the meaning
of this option when that option is also non-empty. If this option is
empty and the _-subcommands_ option is empty too, the namespace will use
the exported commands of the namespace as its command set, dynamically
determining them \(subject to cacheing\) every time the ensemble is
_invoked_. Note that the words in the dictionary values that give
the commands to map to \(as opposed to any arguments to them\) are
always resolved \(if not absolute\) relative to the namespace which is
running [namespace ensemble create] or [namespace ensemble
configure].
-prefixes: This boolean option \(which is on by default\) controls whether
unambiguous prefixes of ensemble commands are recognized as if they
were the fully specified ensemble command names.
-unknown: This provides \(when non-empty\) a partial command to handle
the case where an ensemble subcommand is not recognized and would
otherwise generate an error. When empty \(the default\) an error \(in
the style of _Tcl\_GetIndexFromObj_\) is generated whenever the
ensemble is unable to determine how to implement a particular
subcommand.
> See _Unknown Handler Behaviour_ below for details of how the
ensemble interacts with its unknown handler.
Ensemble creation takes an extra option value pair:
-command: This option allows you to override what the name of the
ensemble to create is.
Ensemble configuring allows the querying of an extra read-only option:
-namespace: This _read-only_ option allows the retrieval of the name
of the namespace that the ensemble was created in.
Given an ensemble command created by the above mechanism, calling the
command will first of all match the subcommand to its implementing
command \(or command/argument list, as derived from the dictionary\) in
a manner that will be recognizably similar to that enforced by
_Tcl\_GetIndexFromObj\(\)_ \(unless the _-unknown_ option override this
behaviour.\) For details of how the ensemble determines what its subcommands are, please see the _-subcommands_ and _-map_ options above.
Then the ensemble command will rewrite the command and arguments so
that the ensemble command and subcommand are replaced by the
implementing command and any specified arguments, with the resulting
word list being fed to _Tcl\_EvalObjv\(\)_ for execution. Note that
this does not increase the stack depth in terms of [uplevel], and
that the implementing command may itself be an ensemble command.
Note that if the namespace for an ensemble is deleted, the ensemble will
also be deleted though there is no lifetime constraint in the other
direction \(i.e. using [rename $theEnsemble {}] will not cause the
namespace to vanish.\) Obviously, any ensemble which has the global
namespace backing it up will have the same natural lifespan as the
hosting Tcl interpreter.
# Unknown Handler Behaviour
If an unknown handler is specified for an ensemble, that handler is called when the ensemble command would otherwise return an error due to it being unable to decide which subcommand to invoke. The exact conditions under which that occurs are controlled by the _-subcommands_, _-map_ and _-prefixes_ options as described above.
To execute the unknown handler, the ensemble mechanism takes the specified -unknown option and appends each argument of the attempted ensemble command invocation \(including the ensemble command itself, expressed as a fully qualified name\). It invokes the resulting command in the scope of the attempted call. If the execution of the unknown handler terminates normally, the ensemble engine reparses the subcommand \(as described below\) and tries to dispatch it again, which is ideal for when the ensemble's configuration has been updated by the unknown subcommand handler. Any other kind of termination of the unknown handler is treated as an error.
The result of the unknown handler is expected to be a list \(it is an error if it is not\). If the list is an empty list, the ensemble command attempts to look up the original subcommand again and, if it is not found this time, an error will be generated just as if the -unknown handler was not there \(i.e. for any particular invokation of an ensemble, its unknown handler will be called at most once.\) This makes it easy for the unknown handler to update the ensemble or its backing namespace so as to provide an implementation of the desired subcommand and reparse.
When the result is a non-empty list, the words of that list are used to replace the ensemble command and subcommand, just as if they had been looked up in the -map. It is up to the unknown handler to supply all namespace qualifiers if the implementing subcommand is not in the namespace of the caller of the ensemble command. Also note that when ensemble commands are chained \(e.g. if you make one of the commands that implement an ensemble subcommand into an ensemble, in a manner similar to the text widget's _tag_ and _mark_ subcommands\) then the rewrite happens in the context of the caller of the outermost ensemble. That is to say that ensembles do not in themselves place any namespace contexts on the Tcl call stack.
Where an empty -unknown handler is given \(the default\), the ensemble command will generate an error message based on the list of commands that the ensemble has defined \(formatted similarly to the error message from _Tcl\_GetIndexFromObj\(\)_\). This is the error that will be thrown when the subcommand is still not recognized during reparsing. It is also an error for an -unknown handler to delete its namespace.
# Examples
namespace eval carrot {
namespace export foo bar potato
namespace ensemble create ;# Creates command ::carrot
proc foo {} {puts 1} ;# Exported
proc bar {} {puts 2} ;# Exported
proc boo {} {puts 3} ;# Not exported
namespace eval turnip { ;# Not exported
namespace export alpha
proc alpha {} {puts 4} ;# Exported
proc beta {} {puts 5} ;# Not exported
namespace ensemble create
} namespace eval potato { ;# Exported
proc north {x} {puts 6,$x} ;# Not exported
proc south {x} {puts 7,$x} ;# Not exported
# Notice we resolve names locally, and Tcl metachars are not special
namespace ensemble create -map {
north {north [$meta[$chars}
} } }carrot foo ;# Prints 1
carrot bar ;# Prints 2
carrot b ;# Also prints 2 ("boo" not exported)
carrot ? ;# ERROR: Alternatives "bar", "foo" and "potato"
carrot potato ;# ERROR: Missing argument
carrot potato ? ;# ERROR: Try "north" instead
carrot potato north ;# Prints 6,[$meta[$chars
carrot turnip alpha ;# ERROR: "turnip" not known
carrot::turnip alpha ;# Prints 4
carrot::turnip::beta ;# Prints 5
rename ::carrot::potato ::spud
spud north ;# Prints 6,[$meta[$chars
spud south ;# ERROR: "south" not known
carrot potato north ;# ERROR: No ::carrot::potato command
# Reconfigure spud; notice we get different name resolution now!
namespace ensemble configure spud -map {
north {puts NORTH} south {puts SOUTH}
}spud north ;# Prints NORTH
spud south ;# Prints SOUTH
namespace delete carrot
spud north ;# ERROR: spud command already deleted
namespace eval A {
proc a args {puts A::a=>$args}
}namespace eval B {
proc b args {puts B::b=>$args}
}# Create an ensemble in the global namespace
namespace ensemble create -command C -map {
eg1 {::A::a foo bar}
eg2 {::B::b 1 2 3}
eg3 ::string
}C eg1 spong ;# Prints A::a=>foo bar spong
C eg2 evil code {[exit]} ;# Prints B::b=>1 2 3 evil code [exit]
C eg3 length qwertyuiop ;# Returns 10
# An example demonstrating the use of -unknown to do delegation
# This uses an ensemble to add a subcommand to a frame
package require Tk
rename [frame .f] _impl.f
namespace ensemble create -command .f -unknown {delegate _impl.f} -map {
flash {flashFrame _impl.f}
}# General delegation framework handler
proc delegate {target ensemble subcmd args} {
# Read the current map
set map [namespace ensemble configure $ensemble -map]
# Update it to include the new subcommand
dict set map $subcmd [list $target $subcmd]
# Install back into the ensemble
namespace ensemble configure $ensemble -map $map
# Result is empty string, so we reparse
return {}
}# Our new subcommand implementation
proc flashFrame w {
set bg [$w cget -background]
foreach colour {black white black white black white} {
$w configure -background $colour
update idletasks
after 150
} $w configure -background $bg
}# Consequences
Many commands in both Tcl and Tk would benefit from leveraging this,
and it would enable straight-forward implementations of things like
[[65]](65.md) in pure Tcl code. It would also make doing things like partial
exposure of ensemble-like commands in safe interpreters much easier.
# Sample Implementation
<http://sf.net/tracker/?func=detail&aid=786509&group\_id=10894&atid=310894># Copyright
This document has been placed in the public domain.
# Acknowledgements
Thanks very much to Joe English, Don Porter and Kevin Kenny for their
suggestions in the development of this TIP. Without them, it would have
been a far worse suggestion. And thanks to Will Duquette for writing a
piece of software \(Snit\) that would benefit immensely from pushing the
ensemble stuff as hard as possible and then a bit. :^\)

TIP: 113Title: Multi-Line Searches in the Text Widget
Version: $Revision: 1.13 $
Author: Vince Darley <vincentdarley@users.sourceforge.net>
State: Final
Type: Project
Vote: Done
Created: 11-Oct-2002
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes enhancing the implementation of the ''$textwidget
search'' subcommand to allow matching of both exact strings and regexp
patterns which span multiple lines, and to allow reporting on all matches.
~ Proposal
If the string/pattern given to the ''search'' subcommand contains
sub-strings/patterns which match newlines, then it will be possible
for that command to return a match which spans multiple lines. Where
a match could occur both within a single line and across multiple
lines, the first such match will be found, and the length of the match
will follow the usual regexp rules, as documented in the regexp man
page. Since the text widget is inherently a line-based container of text, regexp searches will implicitly use the regexp ''-line'' functionality so that ''^'', ''$'' matching beginning and end of any line, and ''.'', ''[^'' will not match a newline (but see the ''-nolinestop'' flag to turn off the latter behaviour).
This can be implemented very efficiently, given
the ''TCL_REG_CANMATCH'' flag supported by the regexp library, with no
impact at all on the speed of matching single lines.
In addition, two new options to the ''search'' subcommand are available:
If the new ''-all'' option is given to the ''search'' subcommand, then all matches within the given range will be reported. This means the return result of the command will be a list of indices, and, if a ''-count var'' option was given, ''var'' will be set to a list of match-index-lengths.
If the new ''-nolinestop'' option is given then regexp searches will allow ''.'' and ''[^'' sequences to match newline characters (which is normally not the case). This is equivalent to ''not'' providing the ''-linestop'' flag to Tcl's ''regexp'' command.
The text widget man page will be updated to reflect the new ''-all'' and ''-nolinestop'' options, and to remove the "single line" caveat.
~ Reference implementation
This is available from:
http://sourceforge.net/tracker/?func=detail&aid=621901&group_id=12997&atid=312997
The patch includes objectification of the entire Text widget, so the multi-line search changes are not obvious to isolate. In fact the changes required are < 100 lines of code (given that the rest has been objectified, that is). Of course one nice side-effect of objectification is that regexp objects used in searches are actually cached, which they previously couldn't be.
Note: this patch has to workaround a crashing bug in Tcl's unicode string manipulation. It would be best if that bug was fixed before applying this patch.
~ Issues
On the implementation side, it might be interesting to abstract the search interface away from the text widget, so that it could in principle be applied to any line-based textual source.
As in the single-line matching implementation in Tcl 8.x, the lack of support for backwards matching in Tcl's regexp library means that backwards matching can only be implemented as repeated forward matches, with a commensurate performance penalty (the solution to which is outside the scope of this tip).
Tk has a curious misfeature that ''$text search -regexp "\n" $pos'' will always fail to match anything. This behaviour will change as a result of this TIP (it will match the first newline after ''$pos''), and any code which somehow depended on that peculiarity will therefore break.
~ Copyright
This document has been placed in the public domain.

# TIP 113: Multi-Line Searches in the Text Widget
Author: Vince Darley <vincentdarley@users.sourceforge.net>
State: Final
Type: Project
Vote: Done
Created: 11-Oct-2002
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes enhancing the implementation of the _$textwidget
search_ subcommand to allow matching of both exact strings and regexp
patterns which span multiple lines, and to allow reporting on all matches.
# Proposal
If the string/pattern given to the _search_ subcommand contains
sub-strings/patterns which match newlines, then it will be possible
for that command to return a match which spans multiple lines. Where
a match could occur both within a single line and across multiple
lines, the first such match will be found, and the length of the match
will follow the usual regexp rules, as documented in the regexp man
page. Since the text widget is inherently a line-based container of text, regexp searches will implicitly use the regexp _-line_ functionality so that _^_, _$_ matching beginning and end of any line, and _._, _[^_ will not match a newline \(but see the _-nolinestop_ flag to turn off the latter behaviour\).
This can be implemented very efficiently, given
the _TCL\_REG\_CANMATCH_ flag supported by the regexp library, with no
impact at all on the speed of matching single lines.
In addition, two new options to the _search_ subcommand are available:
If the new _-all_ option is given to the _search_ subcommand, then all matches within the given range will be reported. This means the return result of the command will be a list of indices, and, if a _-count var_ option was given, _var_ will be set to a list of match-index-lengths.
If the new _-nolinestop_ option is given then regexp searches will allow _._ and _[^_ sequences to match newline characters \(which is normally not the case\). This is equivalent to _not_ providing the _-linestop_ flag to Tcl's _regexp_ command.
The text widget man page will be updated to reflect the new _-all_ and _-nolinestop_ options, and to remove the "single line" caveat.
# Reference implementation
This is available from:
<http://sourceforge.net/tracker/?func=detail&aid=621901&group\_id=12997&atid=312997>
The patch includes objectification of the entire Text widget, so the multi-line search changes are not obvious to isolate. In fact the changes required are < 100 lines of code \(given that the rest has been objectified, that is\). Of course one nice side-effect of objectification is that regexp objects used in searches are actually cached, which they previously couldn't be.
Note: this patch has to workaround a crashing bug in Tcl's unicode string manipulation. It would be best if that bug was fixed before applying this patch.
# Issues
On the implementation side, it might be interesting to abstract the search interface away from the text widget, so that it could in principle be applied to any line-based textual source.
As in the single-line matching implementation in Tcl 8.x, the lack of support for backwards matching in Tcl's regexp library means that backwards matching can only be implemented as repeated forward matches, with a commensurate performance penalty \(the solution to which is outside the scope of this tip\).
Tk has a curious misfeature that _$text search -regexp "\\n" $pos_ will always fail to match anything. This behaviour will change as a result of this TIP \(it will match the first newline after _$pos_\), and any code which somehow depended on that peculiarity will therefore break.
# Copyright
This document has been placed in the public domain.

TIP: 114Title: Eliminate Octal Parsing of Leading Zero Integer Strings
Author: Don Porter <dgp@users.sf.net>
Created: 16-Oct-2007
Type: Project
State: Draft
Vote: Done
Version: $Revision: 2.3 $
Tcl-Version: 9.0
Post-History:
Keywords: octal
~Abstract
This TIP proposes elimination of Tcl's practice of using octal
notation to interpret a string with a leading zero when an
integer value is expected.
~History and Rationale
There are several places in the syntax of several Tcl commands
where an integer value may be accepted. Routines such as
'''Tcl_GetInt()''' perform the task of parsing an integer value
from the string value in these places. Ultimately, these routines
have been built on C standard library functions such as
'''strtol()'''. Due to this implementation choice, Tcl integer
parsing has inherited features from '''strtol()''' including
the feature that a leading zero in a string has been taken
as a signal that the string is an integer value in octal format.
Several programmers and programs have hit this feature by
surprise, resulting in nasty bugs such as:
| % proc m date {
| lassign [split $date -] y m d
| return [string index _JFMAMJJASOND $m]
| }| % m 2007-02-14
| F| % m 2007-12-25
| D| % m 2007-09-26
| bad index "09": must be integer?[+-]integer? or end?[+-]integer? (looks like invalid octal number)
There are very few places in Tcl scripts where this feature
is actually useful. Octal format for integers simply isn't
encountered all that often in most programming tasks tackled by
Tcl scripts. The main counterexample is the use of octal format
integers to describe filesystem permissions on unix systems.
The Tcl commands that operate on filesystem permission values
are '''open''' and '''file attributes''', and it is a simple matter
to directly code them to recognize octal format, rather that have
then rely on octal parsing as a general integer value recognition
feature. On the HEAD, these commands have already been so revised.
With those few cases accounted for, it's been observed that
removing this feature of Tcl integer parsing "will likely fix
more scripts than it breaks."
The opportunity to make this change in Tcl 8.5 arises because
we've already replaced our old parsing routines based on
'''strtol()''' with our own number parser [249].
~ Proposal
Revise all integer parsing in Tcl by making modifications to
the '''TclParseNumber()''' routine. With reference to the
state machine graph in [249], we change the exit edges of
state ''integer[[1]]''. Characters '''0''' - '''7''' and
characters '''8''' - '''9''' should now lead to state ''integer[[4]]'',
so that they continue decimal parsing, and not octal parsing.
The states ''integer[[2]]'' and ''error[[5]]'' will now be
accessible only if the character '''o''' or '''O''' is seen
while in state ''integer[[1]]'' and there will no longer be
any exit from those states when the characters '''.''' or '''e'''
or '''E''' are observed.
This change to '''TclParseNumber()''' is achieved with a
'''#define KILL_OCTAL''' in the file ''tclStrToD.c''.
~ Compatibility
This change is an incompatibility. It's long
been believed that such a change should not happen until Tcl 9
because of this, but over time the consensus belief has developed
that far fewer programs and programmers will be harmed by the
incompatibility than will be helped by removing the misfeature.
That said, the incompatibility is serious. The same string in
the same place in a script can now have a completely different
meaning. Before the change:
| % lindex {a b c d e f g h i j k} 010
| i
After the change:
| % lindex {a b c d e f g h i j k} 010
| k
This is not the usual situation where new feature causes scripts
that were an error to become non-errors -- a compatible change.
This is also not a situation where a change causes legal scripts
to become errors. Such a change would break scripts, but would
at least leave behind scripts that raise noisy errors alerting
about the breakage.
This is the most serious kind of incompatibility, where we replace
a working script with another working script that does something
completely different. An illustration of the problem from Tcl's
own test suite highlights the danger. Some of Tcl's tests in
'''io.test''' depend on the umask value, so that value is captured:
| set umaskValue [exec /bin/sh -c umask]
Note that the shell command '''umask''' returns a mask value as
an integer in octal format. The test suite has relied on Tcl's
built-in ability to recognize this format, and the expected
result of test '''io-40.3''' has been computed:
| format %04o [expr {0666 & ~$umaskValue}]
After the proposed change, ''$umaskValue'' is treated as a decimal
number, and the wrong expected result is computed. (This test
has already been updated on the HEAD to avoid such problems.)
It is not difficult to imagine more serious problems in scripts
that make use of the result returned by the shell command '''umask'''
where a file might be created or modified with completely unintended
permissions as a result of the proposed change. Such scripts
might easily raise security concerns.
Even in the light of the judgment that such (hopefully rare) compatibility
issues are acceptable in exchange for the benefits of purging the
misfeature, we really ought to consider seriously how we can alert
those migrating to Tcl 8.5 to this possibility and to the need to
examine their scripts for this issue.
Besides the impact on Tcl commands, this change may also cause
incompatibilities in extensions, to the extent their commands
rely on Tcl's integer parsing to support octal notation.
~ Rejected Alternatives
Motivated largely by the serious incompatibilities lurking here,
a few people have suggested that some means be provided to toggle
Tcl's integer parsing behavior between two modes, one which
recognizes octal and one which does not. This idea appears inspired
in part by the '''::tcl_precision''' variable, which has long
exercised control over Tcl's floating point number formatting.
While the motivation may be well-intended, this proposal is basically
unworkable, and can't really help anybody. The point of the proposed
change is to make simple code work as simple coders expect it to.
Our original example proc can already be corrected like so:
| proc m date {
| scan $date %d-%d-%d y m d
| return [string index _JFMAMJJASOND $m]
| }
The point of this proposal is to make the original code just work.
It doesn't help to offer this complexity as a solution:
| % proc m date {
| set mode [tcl::unsupported::octal]
| tcl::unsupported::octal off
| lassign [split $date -] y m d
| set result [string index _JFMAMJJASOND $m]
| tcl::unsupported::octal $mode
| return $result
| }
No one would choose that over just fixing the code
to use '''scan''', in which case this proposal won't be needed.
Also, this kind of management of a shared mode setting cannot
(easily and cheaply) be avoided, because at the point we need
to control the Tcl number parser, the most specific context
we have is the thread, so the mode has to be set thread-wide.
Likewise, the (hopefully rare) set of scripts that would actually
want to turn octal parsing back on are not going to announce themselves.
In order to know that
| tcl::unsupported::octal on
needs to be added to a script to make it function correctly,
some kind of audit has to reach that conclusion, and once that
conclusion is reached and the issues are understood, it's just
as easy to insert '''scan %o''' in the proper places as it would
be to insert the '''tcl::unsupported::octal''' stopgap.
In short, any coder finding themselves in a position to consider
using a '''tcl::unsupported::octal''' tool, would quickly decide
not to use it in favor of just fixing their code. Thus users
of this feature are mythical, and it will not be implemented.
~ Note
This TIP has been ''explicitly'' rejected as a feature for Tcl 8.5.
Consensus was that the type of breakage it inherently induces is not
acceptable in a minor version change.
~Copyright
This document is placed in the public domain.

# TIP 114: Eliminate Octal Parsing of Leading Zero Integer Strings
Author: Don Porter <dgp@users.sf.net>
Created: 16-Oct-2007
Type: Project
State: Draft
Vote: Done
Tcl-Version: 9.0
Post-History:
Keywords: octal
-----# Abstract
This TIP proposes elimination of Tcl's practice of using octal
notation to interpret a string with a leading zero when an
integer value is expected.
# History and Rationale
There are several places in the syntax of several Tcl commands
where an integer value may be accepted. Routines such as
**Tcl\_GetInt\(\)** perform the task of parsing an integer value
from the string value in these places. Ultimately, these routines
have been built on C standard library functions such as
**strtol\(\)**. Due to this implementation choice, Tcl integer
parsing has inherited features from **strtol\(\)** including
the feature that a leading zero in a string has been taken
as a signal that the string is an integer value in octal format.
Several programmers and programs have hit this feature by
surprise, resulting in nasty bugs such as:
% proc m date {
lassign [split $date -] y m d
return [string index _JFMAMJJASOND $m]
} % m 2007-02-14
F % m 2007-12-25
D % m 2007-09-26
bad index "09": must be integer?[+-]integer? or end?[+-]integer? (looks like invalid octal number)
There are very few places in Tcl scripts where this feature
is actually useful. Octal format for integers simply isn't
encountered all that often in most programming tasks tackled by
Tcl scripts. The main counterexample is the use of octal format
integers to describe filesystem permissions on unix systems.
The Tcl commands that operate on filesystem permission values
are **open** and **file attributes**, and it is a simple matter
to directly code them to recognize octal format, rather that have
then rely on octal parsing as a general integer value recognition
feature. On the HEAD, these commands have already been so revised.
With those few cases accounted for, it's been observed that
removing this feature of Tcl integer parsing "will likely fix
more scripts than it breaks."
The opportunity to make this change in Tcl 8.5 arises because
we've already replaced our old parsing routines based on
**strtol\(\)** with our own number parser [[249]](249.md).
# Proposal
Revise all integer parsing in Tcl by making modifications to
the **TclParseNumber\(\)** routine. With reference to the
state machine graph in [[249]](249.md), we change the exit edges of
state _integer[[1]](1.md)_. Characters **0** - **7** and
characters **8** - **9** should now lead to state _integer[[4]](4.md)_,
so that they continue decimal parsing, and not octal parsing.
The states _integer[[2]](2.md)_ and _error[[5]](5.md)_ will now be
accessible only if the character **o** or **O** is seen
while in state _integer[[1]](1.md)_ and there will no longer be
any exit from those states when the characters **.** or **e**
or **E** are observed.
This change to **TclParseNumber\(\)** is achieved with a
**\#define KILL\_OCTAL** in the file _tclStrToD.c_.
# Compatibility
This change is an incompatibility. It's long
been believed that such a change should not happen until Tcl 9
because of this, but over time the consensus belief has developed
that far fewer programs and programmers will be harmed by the
incompatibility than will be helped by removing the misfeature.
That said, the incompatibility is serious. The same string in
the same place in a script can now have a completely different
meaning. Before the change:
% lindex {a b c d e f g h i j k} 010
i
After the change:
% lindex {a b c d e f g h i j k} 010
k
This is not the usual situation where new feature causes scripts
that were an error to become non-errors -- a compatible change.
This is also not a situation where a change causes legal scripts
to become errors. Such a change would break scripts, but would
at least leave behind scripts that raise noisy errors alerting
about the breakage.
This is the most serious kind of incompatibility, where we replace
a working script with another working script that does something
completely different. An illustration of the problem from Tcl's
own test suite highlights the danger. Some of Tcl's tests in
**io.test** depend on the umask value, so that value is captured:
set umaskValue [exec /bin/sh -c umask]
Note that the shell command **umask** returns a mask value as
an integer in octal format. The test suite has relied on Tcl's
built-in ability to recognize this format, and the expected
result of test **io-40.3** has been computed:
format %04o [expr {0666 & ~$umaskValue}]
After the proposed change, _$umaskValue_ is treated as a decimal
number, and the wrong expected result is computed. \(This test
has already been updated on the HEAD to avoid such problems.\)
It is not difficult to imagine more serious problems in scripts
that make use of the result returned by the shell command **umask**
where a file might be created or modified with completely unintended
permissions as a result of the proposed change. Such scripts
might easily raise security concerns.
Even in the light of the judgment that such \(hopefully rare\) compatibility
issues are acceptable in exchange for the benefits of purging the
misfeature, we really ought to consider seriously how we can alert
those migrating to Tcl 8.5 to this possibility and to the need to
examine their scripts for this issue.
Besides the impact on Tcl commands, this change may also cause
incompatibilities in extensions, to the extent their commands
rely on Tcl's integer parsing to support octal notation.
# Rejected Alternatives
Motivated largely by the serious incompatibilities lurking here,
a few people have suggested that some means be provided to toggle
Tcl's integer parsing behavior between two modes, one which
recognizes octal and one which does not. This idea appears inspired
in part by the **::tcl\_precision** variable, which has long
exercised control over Tcl's floating point number formatting.
While the motivation may be well-intended, this proposal is basically
unworkable, and can't really help anybody. The point of the proposed
change is to make simple code work as simple coders expect it to.
Our original example proc can already be corrected like so:
proc m date {
scan $date %d-%d-%d y m d
return [string index _JFMAMJJASOND $m]
}
The point of this proposal is to make the original code just work.
It doesn't help to offer this complexity as a solution:
% proc m date {
set mode [tcl::unsupported::octal]
tcl::unsupported::octal off
lassign [split $date -] y m d
set result [string index _JFMAMJJASOND $m]
tcl::unsupported::octal $mode
return $result
}
No one would choose that over just fixing the code
to use **scan**, in which case this proposal won't be needed.
Also, this kind of management of a shared mode setting cannot
\(easily and cheaply\) be avoided, because at the point we need
to control the Tcl number parser, the most specific context
we have is the thread, so the mode has to be set thread-wide.
Likewise, the \(hopefully rare\) set of scripts that would actually
want to turn octal parsing back on are not going to announce themselves.
In order to know that
tcl::unsupported::octal on
needs to be added to a script to make it function correctly,
some kind of audit has to reach that conclusion, and once that
conclusion is reached and the issues are understood, it's just
as easy to insert **scan %o** in the proper places as it would
be to insert the **tcl::unsupported::octal** stopgap.
In short, any coder finding themselves in a position to consider
using a **tcl::unsupported::octal** tool, would quickly decide
not to use it in favor of just fixing their code. Thus users
of this feature are mythical, and it will not be implemented.
# Note
This TIP has been _explicitly_ rejected as a feature for Tcl 8.5.
Consensus was that the type of breakage it inherently induces is not
acceptable in a minor version change.
# Copyright
This document is placed in the public domain.

TIP: 115Title: Making Tcl Truly 64-Bit Ready
Version: $Revision: 1.3 $
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Draft
Type: Project
Vote: Pending
Created: 23-Oct-2002
Post-History:
Tcl-Version: 9.0
~ Abstract
This TIP proposes changes to Tcl to make it operate more effectively
on 64-bit systems.
~ Rationale
It is a fact of life that 64-bit platforms are becoming more common.
While once the assumption that virtually everything was a 32-bit
machine (where not smaller) was valid, this is no longer the case.
Particularly on modern supercomputers (though increasingly in
workstations and high-end desktop systems too), the amount of memory
that the machine contains is exceeding 2GB, and the need to address
very large amounts of memory is certainly there in scientific and
engineering applications. And where they lead, consumer systems will
probably follow too.
At the moment, Tcl is ill-prepared for this. In particular, the type
used for expressing sizes of entities in Tcl (whether strings, lists
or undifferentiated blocks of memory) is ''int'' (and cannot be made
into an ''unsigned int'' in most of those places where it is not
already an unsigned value) but on the majority of 64-bit platforms
this is still a 32-bit type, which is a major restriction. However,
on the vast majority of those platforms ''long'' is a 64-bit type, and
so a suitable replacement. (The exceptions to this are the Alpha - but
that is unusual in that both ''int'' and ''long'' are 64-bit types
there, meaning that the platform will be unaffected by such an
alteration - and Win64, which has a 32-bit ''long'' but 64-bit
pointers.)
Luckily, standards like POSIX have already been dealing with this
problem before us, and the types ''size_t'' (which is unsigned) and
''ssize_t'' (which is signed) exist for the sorts of uses we're
interested in (i.e. they are both the same size as each other, and
''size_t'' is large enough to describe the size of any allocatable
memory chunk.)
~ Details of Changes
The key changes will be to change the lengths of the following types
from ''int'' to ''ssize_t'' in all appropriate places, and ''unsigned
int'' to ''size_t'' likewise (mainly in memory allocation routines.)
* ''Tcl_Obj'' - the ''length'' member. (Potentially the ''refCount''
member needs updating as well, but that's less critical.)
* ''Tcl_SavedResult'' - the ''appendAvl'' and ''appendUsed'' members.
* ''Tcl_DString'' - the ''length'' and ''spaceAvl'' members.
* ''Tcl_Token'' - the ''size'' and ''numComponents'' members.
* ''Tcl_Parse'' - the ''commentSize'', ''commandSize'', numWords'',
''numTokens'' and ''tokensAvailable'' members.
* ''CompiledLocal'' - the ''nameLength'' member.
* ''Interp'' - the ''appendAvl'', ''appendUsed'' and ''termOffset''
members.
* ''List'' - the ''maxElemCount'' and ''elemCount'' members.
* ''ByteArray'' - the ''used'' and ''allocated'' members.
* ''SortElement'' - the ''count'' member.
* ''SortInfo'' - the ''index'' member.
* ''CopyState'' - the ''toRead'' and ''total'' members.
* ''GetsState'' - the ''rawRead'', ''bytesWrote'', ''charsWrote'' and
''totalChars'' members.
* ''ParseInfo'' - the ''size'' member.
* ''String'' - the ''numChars'' member (see also the ''TestString''
structure.)
Changes to the bytecode-related structures might be worthwhile doing
too, though there are more backward-compatibility issues there.
These changes will force many of the types used in the public API to
change as well. Notable highlights:
* ''Tcl_Alloc'' will now take an ''size_t''.
* ''Tcl_GetByteArrayFromObj'' will now take a pointer to a ''ssize_t''.
* ''Tcl_GetStringFromObj'' will now take a pointer to a ''ssize_t''.
* ''Tcl_ListObjLength'' will now take a pointer to a ''ssize_t''.
* ''Tcl_GetUnicodeFromObj'' will now take a pointer to a ''ssize_t''.
In the internal API, the following notable change will happen:
* ''TclGetIntForIndex'' will now take a pointer to a ''ssize_t''.
There are probably other similar API changes required.
~ What This TIP Does Not Do
This TIP does not rearrange structure orderings. Although this would
be very useful for some common structures (notably ''Tcl_Obj'') if the
common arithmetic types were smaller than the word size, it turns out
that the changes in types required to deal with larger entities will
make these rearrangements largely unnecessary and/or pointless.
(Inefficiency in statically-allocated structures won't matter as the
number of instances will remain comparatively small, even in very
large programs.) Once the changes are applied, there is typically at
most a single ''int'' field per structure, usually holding either a
reference count, a set of flags, or a Tcl result code.
It should also be noted that all structures are always going to be
correctly aligned internally as we never use C's bitfield support, so
structure alignment is purely an issue of efficiency, and not of correct
access to the fields.
~ Copyright
This document has been placed in the public domain.

# TIP 115: Making Tcl Truly 64-Bit Ready
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Draft
Type: Project
Vote: Pending
Created: 23-Oct-2002
Post-History:
Tcl-Version: 9.0
-----# Abstract
This TIP proposes changes to Tcl to make it operate more effectively
on 64-bit systems.
# Rationale
It is a fact of life that 64-bit platforms are becoming more common.
While once the assumption that virtually everything was a 32-bit
machine \(where not smaller\) was valid, this is no longer the case.
Particularly on modern supercomputers \(though increasingly in
workstations and high-end desktop systems too\), the amount of memory
that the machine contains is exceeding 2GB, and the need to address
very large amounts of memory is certainly there in scientific and
engineering applications. And where they lead, consumer systems will
probably follow too.
At the moment, Tcl is ill-prepared for this. In particular, the type
used for expressing sizes of entities in Tcl \(whether strings, lists
or undifferentiated blocks of memory\) is _int_ \(and cannot be made
into an _unsigned int_ in most of those places where it is not
already an unsigned value\) but on the majority of 64-bit platforms
this is still a 32-bit type, which is a major restriction. However,
on the vast majority of those platforms _long_ is a 64-bit type, and
so a suitable replacement. \(The exceptions to this are the Alpha - but
that is unusual in that both _int_ and _long_ are 64-bit types
there, meaning that the platform will be unaffected by such an
alteration - and Win64, which has a 32-bit _long_ but 64-bit
pointers.\)
Luckily, standards like POSIX have already been dealing with this
problem before us, and the types _size\_t_ \(which is unsigned\) and
_ssize\_t_ \(which is signed\) exist for the sorts of uses we're
interested in \(i.e. they are both the same size as each other, and
_size\_t_ is large enough to describe the size of any allocatable
memory chunk.\)
# Details of Changes
The key changes will be to change the lengths of the following types
from _int_ to _ssize\_t_ in all appropriate places, and _unsigned
int_ to _size\_t_ likewise \(mainly in memory allocation routines.\)
* _Tcl\_Obj_ - the _length_ member. \(Potentially the _refCount_
member needs updating as well, but that's less critical.\)
* _Tcl\_SavedResult_ - the _appendAvl_ and _appendUsed_ members.
* _Tcl\_DString_ - the _length_ and _spaceAvl_ members.
* _Tcl\_Token_ - the _size_ and _numComponents_ members.
* _Tcl\_Parse_ - the _commentSize_, _commandSize_, numWords_,
_numTokens_ and _tokensAvailable_ members.
* _CompiledLocal_ - the _nameLength_ member.
* _Interp_ - the _appendAvl_, _appendUsed_ and _termOffset_
members.
* _List_ - the _maxElemCount_ and _elemCount_ members.
* _ByteArray_ - the _used_ and _allocated_ members.
* _SortElement_ - the _count_ member.
* _SortInfo_ - the _index_ member.
* _CopyState_ - the _toRead_ and _total_ members.
* _GetsState_ - the _rawRead_, _bytesWrote_, _charsWrote_ and
_totalChars_ members.
* _ParseInfo_ - the _size_ member.
* _String_ - the _numChars_ member \(see also the _TestString_
structure.\)
Changes to the bytecode-related structures might be worthwhile doing
too, though there are more backward-compatibility issues there.
These changes will force many of the types used in the public API to
change as well. Notable highlights:
* _Tcl\_Alloc_ will now take an _size\_t_.
* _Tcl\_GetByteArrayFromObj_ will now take a pointer to a _ssize\_t_.
* _Tcl\_GetStringFromObj_ will now take a pointer to a _ssize\_t_.
* _Tcl\_ListObjLength_ will now take a pointer to a _ssize\_t_.
* _Tcl\_GetUnicodeFromObj_ will now take a pointer to a _ssize\_t_.
In the internal API, the following notable change will happen:
* _TclGetIntForIndex_ will now take a pointer to a _ssize\_t_.
There are probably other similar API changes required.
# What This TIP Does Not Do
This TIP does not rearrange structure orderings. Although this would
be very useful for some common structures \(notably _Tcl\_Obj_\) if the
common arithmetic types were smaller than the word size, it turns out
that the changes in types required to deal with larger entities will
make these rearrangements largely unnecessary and/or pointless.
\(Inefficiency in statically-allocated structures won't matter as the
number of instances will remain comparatively small, even in very
large programs.\) Once the changes are applied, there is typically at
most a single _int_ field per structure, usually holding either a
reference count, a set of flags, or a Tcl result code.
It should also be noted that all structures are always going to be
correctly aligned internally as we never use C's bitfield support, so
structure alignment is purely an issue of efficiency, and not of correct
access to the fields.
# Copyright
This document has been placed in the public domain.

TIP: 116Title: More Safety for Large Images
Version: $Revision: 1.6 $
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 28-Oct-2002
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP alters the C API for Tk's images so that failures to allocate
sufficient memory for a large image can be handled more gracefully
than a straight ''panic()''.
~ Rationale
Tk's image mechanism is nice and flexible, but it can run into
problems with a large image. If we consider a square image that is
2000 pixels on each side (such sizes of images are becoming more
common with the increasing popularity and sophistication of digital
photography), we find it requires about 16MB of memory to load (4
million pixels, four bytes per pixel) but obviously just because an
application fails to load that image (or something even larger), it
doesn't mean that the best course of action is a ''panic()''-induced
crash. Instead, a graceful failure back to the Tcl interpreter would
allow for scripts to find a way to report this low-memory situation in
a way that users can understand.
The problem with this is that for many of the routes through the Tk
image API, there is no way to report a memory allocation failure;
currently, the only failure mode is total. This TIP changes this.
~ Proposed Changes
I propose making the following functions that currently return
''void'' return ''int'' instead and that they should additionally take
a standard ''Tcl_Interp *interp'' argument to allow an error message
describing the failure (currently only due to insufficient memory) to
be added to the current interpreter. It will not be an error for the
''interp'' argument to be ''NULL'', though it will be up to the caller
to guess why the failure happened.
* ''Tk_PhotoExpand'' will become:
|int
|Tk_PhotoExpand(Tcl_Interp *interp, Tk_PhotoHandle handle,
| int width, int height)
* ''Tk_PhotoPutBlock'' will become:
|int
|Tk_PhotoPutBlock(Tcl_Interp *interp, Tk_PhotoHandle handle,
| Tk_PhotoImageBlock *blockPtr,
| int x, int y, int width, int height, int compRule)
* ''Tk_PhotoPutZoomedBlock'' will become:
|int
|Tk_PhotoPutZoomedBlock(Tcl_Interp *interp, Tk_PhotoHandle handle,
| Tk_PhotoImageBlock *blockPtr,
| int x, int y, int width, int height,
| int zoomX, int zoomY, int subsampleX, int subsampleY,
| int compRule)
* ''Tk_PhotoSetSize'' will become:
|int
|Tk_PhotoSetSize(Tcl_Interp *interp, Tk_PhotoHandle handle,
| int width, int height)
Also note that as a consequence of this, some image-related Tk
commands will also gain additional error return situations. Since
these all trigger abnormal process termination (and potentially a
core-dump too) at the moment, this change in behaviour is believed to
be wholly beneficial.
~ Backward Compatibility
This TIP also proposes a backward compatibility interface, so that
extensions need not be heavily modified to work with new versions of
Tk. This is done by leaving backwardly-compatible functions in the
old locations in Tk's stub table and adding a ''#define'' to allow
selection of the old API with the standard names. I propose doing this
when the symbol ''USE_PANIC_ON_PHOTO_ALLOC_FAILURE'' is defined; like
this, extension authors can switch from compiling with Tk 8.4 to using
later versions by adding just one flag to their makefile (or other build
script/environment.)
Note that this interacts with the backward-compatability interface
defined in [98]; if that is enabled, the back-compat interface defined
here is enabled as well.
~ Sample Implementation
http://sf.net/tracker/?func=detail&aid=646382&group_id=12997&atid=312997~ Copyright
This document has been placed in the public domain.

# TIP 116: More Safety for Large Images
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 28-Oct-2002
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP alters the C API for Tk's images so that failures to allocate
sufficient memory for a large image can be handled more gracefully
than a straight _panic\(\)_.
# Rationale
Tk's image mechanism is nice and flexible, but it can run into
problems with a large image. If we consider a square image that is
2000 pixels on each side \(such sizes of images are becoming more
common with the increasing popularity and sophistication of digital
photography\), we find it requires about 16MB of memory to load \(4
million pixels, four bytes per pixel\) but obviously just because an
application fails to load that image \(or something even larger\), it
doesn't mean that the best course of action is a _panic\(\)_-induced
crash. Instead, a graceful failure back to the Tcl interpreter would
allow for scripts to find a way to report this low-memory situation in
a way that users can understand.
The problem with this is that for many of the routes through the Tk
image API, there is no way to report a memory allocation failure;
currently, the only failure mode is total. This TIP changes this.
# Proposed Changes
I propose making the following functions that currently return
_void_ return _int_ instead and that they should additionally take
a standard _Tcl\_Interp \*interp_ argument to allow an error message
describing the failure \(currently only due to insufficient memory\) to
be added to the current interpreter. It will not be an error for the
_interp_ argument to be _NULL_, though it will be up to the caller
to guess why the failure happened.
* _Tk\_PhotoExpand_ will become:
int
Tk_PhotoExpand(Tcl_Interp *interp, Tk_PhotoHandle handle,
int width, int height)
* _Tk\_PhotoPutBlock_ will become:
int
Tk_PhotoPutBlock(Tcl_Interp *interp, Tk_PhotoHandle handle,
Tk_PhotoImageBlock *blockPtr,
int x, int y, int width, int height, int compRule)
* _Tk\_PhotoPutZoomedBlock_ will become:
int
Tk_PhotoPutZoomedBlock(Tcl_Interp *interp, Tk_PhotoHandle handle,
Tk_PhotoImageBlock *blockPtr,
int x, int y, int width, int height,
int zoomX, int zoomY, int subsampleX, int subsampleY,
int compRule)
* _Tk\_PhotoSetSize_ will become:
int
Tk_PhotoSetSize(Tcl_Interp *interp, Tk_PhotoHandle handle,
int width, int height)
Also note that as a consequence of this, some image-related Tk
commands will also gain additional error return situations. Since
these all trigger abnormal process termination \(and potentially a
core-dump too\) at the moment, this change in behaviour is believed to
be wholly beneficial.
# Backward Compatibility
This TIP also proposes a backward compatibility interface, so that
extensions need not be heavily modified to work with new versions of
Tk. This is done by leaving backwardly-compatible functions in the
old locations in Tk's stub table and adding a _\#define_ to allow
selection of the old API with the standard names. I propose doing this
when the symbol _USE\_PANIC\_ON\_PHOTO\_ALLOC\_FAILURE_ is defined; like
this, extension authors can switch from compiling with Tk 8.4 to using
later versions by adding just one flag to their makefile \(or other build
script/environment.\)
Note that this interacts with the backward-compatability interface
defined in [[98]](98.md); if that is enabled, the back-compat interface defined
here is enabled as well.
# Sample Implementation
<http://sf.net/tracker/?func=detail&aid=646382&group\_id=12997&atid=312997># Copyright
This document has been placed in the public domain.

TIP: 117Title: Object Type Introspection
Version: $Revision: 1.9 $
Author: Peter Spjuth <peter.spjuth@space.se>
State: Withdrawn
Type: Project
Vote: Pending
Created: 01-Nov-2002
Post-History:
Tcl-Version: 8.5
Obsoleted-By: 214
~ Abstract
This TIP proposes to add a command to give information
on the current internal representation of an object.
~ Rationale
When trying to understand the internals of objects and when trying to
track down shimmering problems it is very helpful to be able to see
what type an object currently has.
~ Specification
A new package called "Debug" is created as part of the core.
A new command, ''objtype'', is added to the Debug package.
The command has the syntax:
> '''Debug::objtype''' ?''value''?
If no value is submitted, a list of all currently registered object
types are returned.
If a value is submitted, it returns a list where the first element
is the name of value's object type, or "none" if there is no
internal representation. The second element is a boolean stating
if the object has a valid string representation.
Examples:
|% Debug::objtype
|boolean index double end-offset wideInt list cmdName bytecode
|nsName procbody bytearray int {array search} string
|% set apa hejsan
|hejsan
|% Debug::objtype $apa
|none 1
|% string length $apa
|6|% Debug::objtype $apa
|string 1
|% regexp $apa hoppsan
|0|% Debug::objtype $apa
|regexp 1
|% Debug::objtype [expr 1+1]
|int 0
~ Discussion
The first proposal was to add this as a subcommand to info. This
was considered bad since this should be a pure debug command
and should never be used in live code. Thus the proposition
came up to create a Debug package and put it in there to emphasise
its debug nature. The Debug package can also be extended in the
future with other utilities.
~ Reference Implementation
|static int
|ObjTypeCmd(dummy, interp, objc, objv)
| ClientData dummy; /* Not used. */
| Tcl_Interp *interp; /* Current interpreter. */
| int objc; /* Number of arguments. */
| Tcl_Obj *CONST objv[]; /* Argument objects. */
|{| Tcl_Obj *listPtr;
|| if ((objc != 2) && (objc != 3)) {
| Tcl_WrongNumArgs(interp, 2, objv, "?value?");
| return TCL_ERROR;
| }|| listPtr = Tcl_NewListObj(0, (Tcl_Obj **) NULL);
|| if (objc == 2) {
| Tcl_AppendAllObjTypes(interp, listPtr);
| } else {
| Tcl_ListObjAppendElement(interp, listPtr, Tcl_NewStringObj(
| (objv[2]->typePtr != NULL) ? objv[2]->typePtr->name : "none", -1));
| Tcl_ListObjAppendElement(interp, listPtr,
| Tcl_NewBooleanObj(objv[2]->bytes != NULL));
| }| Tcl_SetObjResult(interp, listPtr);
| return TCL_OK;
|}~ Future possibilities
For further object introspection the command can easily be
extended by options.
~ Copyright
This document has been placed in the public domain.

# TIP 117: Object Type Introspection
Author: Peter Spjuth <peter.spjuth@space.se>
State: Withdrawn
Type: Project
Vote: Pending
Created: 01-Nov-2002
Post-History:
Tcl-Version: 8.5
Obsoleted-By: 214
-----# Abstract
This TIP proposes to add a command to give information
on the current internal representation of an object.
# Rationale
When trying to understand the internals of objects and when trying to
track down shimmering problems it is very helpful to be able to see
what type an object currently has.
# Specification
A new package called "Debug" is created as part of the core.
A new command, _objtype_, is added to the Debug package.
The command has the syntax:
> **Debug::objtype** ?_value_?
If no value is submitted, a list of all currently registered object
types are returned.
If a value is submitted, it returns a list where the first element
is the name of value's object type, or "none" if there is no
internal representation. The second element is a boolean stating
if the object has a valid string representation.
Examples:
% Debug::objtype
boolean index double end-offset wideInt list cmdName bytecode
nsName procbody bytearray int {array search} string
% set apa hejsan
hejsan
% Debug::objtype $apa
none 1
% string length $apa
6% Debug::objtype $apa
string 1
% regexp $apa hoppsan
0% Debug::objtype $apa
regexp 1
% Debug::objtype [expr 1+1]
int 0
# Discussion
The first proposal was to add this as a subcommand to info. This
was considered bad since this should be a pure debug command
and should never be used in live code. Thus the proposition
came up to create a Debug package and put it in there to emphasise
its debug nature. The Debug package can also be extended in the
future with other utilities.
# Reference Implementation
static int
ObjTypeCmd(dummy, interp, objc, objv)
ClientData dummy; /* Not used. */
Tcl_Interp *interp; /* Current interpreter. */
int objc; /* Number of arguments. */
Tcl_Obj *CONST objv[]; /* Argument objects. */
{ Tcl_Obj *listPtr;
if ((objc != 2) && (objc != 3)) {
Tcl_WrongNumArgs(interp, 2, objv, "?value?");
return TCL_ERROR;
} listPtr = Tcl_NewListObj(0, (Tcl_Obj **) NULL);
if (objc == 2) {
Tcl_AppendAllObjTypes(interp, listPtr);
} else {
Tcl_ListObjAppendElement(interp, listPtr, Tcl_NewStringObj(
(objv[2]->typePtr != NULL) ? objv[2]->typePtr->name : "none", -1));
Tcl_ListObjAppendElement(interp, listPtr,
Tcl_NewBooleanObj(objv[2]->bytes != NULL));
} Tcl_SetObjResult(interp, listPtr);
return TCL_OK;
}# Future possibilities
For further object introspection the command can easily be
extended by options.
# Copyright
This document has been placed in the public domain.

TIP: 118Title: Enhance [file attributes] and [file copy] on Mac OS X & BSD
Version: $Revision: 1.7 $
Author: Daniel A. Steffen <das@users.sourceforge.net>
State: Final
Type: Project
Vote: Done
Created: 01-Nov-2002
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes a set of changes to ''[file attributes]'' and
''[file copy]'' to make them function better on MacOS X and other
BSD-Unix systems.
~ Summary of Proposed Changes
This TIP proposes five sets of changes to ''[file attributes]'' and
''[file copy]'':
1. Add support for the Macintosh and Windows specific flag ''[file
attributes -readonly]'' to Unixes that support the ''chflags()''
API, to allow the ''user immutable'' flag to be accessed &
modified from Tcl.
2. Add support for the Macintosh specific flags ''[file attributes
-readonly -creator -type -hidden]'' to Mac OS X via the POSIX
level API ''getattrlist()''.
3. Add a new flag ''[file attributes -rsrclength]'' to Mac OS X and
Mac OS 9 that gives the length of the resource fork of a file; 0
is the only value that this attribute can be set to, which strips
the resource fork off a file.
4. Change ''[file attributes]'' to return the list of attributes that
can be retrieved without error for a given file, instead of
aborting the whole command when any error occurs.
5. Enhance ''[file copy]'' on Mac OS X (more precisely, the native
file-system ''Tcl_FSCopyFile'') to copy finder attributes (i.e.
''-readonly -creator -type -hidden'') and resource forks
transparently.
~ Rationale
There is currently no way to access and modify HFS file-system metadata
from Tcl on Mac OS X whereas Tcl on Mac OS 9 (or Classic) on the same
Macintosh has that capability. Worse, ''[file copy]'' (and
potentially even ''[file rename]'' if it results in a copy) on Mac OS
X can be a destructive operation at present if it operates on a file
that has essential data in its resource fork or its HFS metadata.
This again in contrast to the same operations in Tcl on Mac OS 9 where
this information is preserved. This TIP seeks to rectify these
asymmetries in order to better hide such file-system related platform
specificities from the scripter.
~ Details
Additional information & examples:
1. Unix versions that support ''chflags()'' include BSD >= 4.4 and
Darwin/Mac OS X (where ''user immutable'' is the flag
corresponding to the file locked state on the HFS file-system,
which is what ''[file attributes -readonly]'' controls on Mac OS
9).
2. The use of ''getattrlist()'' does not require linking with Carbon
and thus allows access to HFS file-system metadata from Tcl on pure
open-source Darwin systems (which is something no other scripting
language can claim at present).
3. The new attribute ''-rsrclength'' is useful to check whether a
file has a resource fork and to calculate total file size on Mac
OS 9 and X (note that ''[file size]'' returns the size of the data
fork only). Stripping a file's resource fork (by setting
''-rsrclength'' to 0) is a common operation on Mac OS when dealing
with files that are destined for other platforms. This is a
feature that has been requested several times and given that it
ties in well with the implementation of the other new attributes
it comes at essentially no additional cost.
|% file attributes test
|-group admin -owner steffen -permissions 00644 -readonly 0 -creator Doug -type RSRC -hidden 0 -rsrclength 314
|% file attributes test -rsrclength 5
|setting nonzero rsrclength not supported
|% file attributes test -rsrclength 0
|% file attributes test
|-group admin -owner steffen -permissions 00644 -readonly 0 -creator Doug -type RSRC -hidden 0 -rsrclength 0
4. On Mac OS X, trying to retrieve the new attributes ''-creator
-type -rsrclength'' fails on non-regular files & directories (and
on any file located on a non-HFS file-system that doesn't support
''getattrlist()''). Returning only the list of attributes that
are valid seems like much more sensible behaviour in this case
than failing with an error and not returning anything. In the
case where no valid attributes can be retrieved at all, the error
returned by the last attempt is passed upstream, to preserve
existing error handling. This proposed change in behaviour of
''[file attributes]'' seems necessary to allow the command to
continue to work in a consistent way on all inputs and on all
platforms; it should not impact existing code since for current
attributes, failure to retrieve any one attribute is equivalent to
failure to retrieve all attributes.
|% close [open test w]
|% file attributes test
|-group admin -owner steffen -permissions 00644 -readonly 0 -creator {} -type {} -hidden 0 -rsrclength 0
|% file delete test
|% file mkdir test
|% file attributes test
|-group admin -owner steffen -permissions 040755 -readonly 0 -hidden 0
|% file delete test
5. Unlike the Finder and other HFS aware tools on Mac OS X, Tcl
currently ignores HFS metadata and the resource fork, which will
undoubtedly surprise scripters unpleasantly. ''[file copy]''
should hide such platform specificities and copy a file in the
same way as the Finder:
|% file attributes test
|-group admin -owner steffen -permissions 00644 -readonly 0 -creator Doug -type RSRC -hidden 0 -rsrclength 314
|% file copy test test1
|% file attributes test1
|-group admin -owner steffen -permissions 00644 -readonly 0 -creator Doug -type RSRC -hidden 0 -rsrclength 314
|% file delete test1
~ Comments
Additional implementation details:
* To support the new attributes ''-creator -type'', routines to
convert from numerical OSTypes (''u_int32_t'') to the usual four
character human readable format have been adapted from
''mac/tclMacResource.c''; the new versions accept/return strings of
length 0-4 unlike the originals that only dealt with length 4.
This is important because creator/type 0 (i.e. ''-creator {} -type
{}'') is common on Mac OS X. The Mac OS 9 implementation of the
OSType string representation code has been modified accordingly by
adding support for strings of length 0-4 and missing
''UtfToExternal/ExternalToUtf'' conversions.
* ''macRoman'' is the encoding used for the string representation of
OSTypes, for consistency with Tcl on Mac OS 9 as well as with
common Mac OS X tools such as Resorcerer & SuperGetInfo that all
use ''macRoman'' to display creator/type codes; this encoding is
probably what most people would expect. It's unfortunate that this
means that use of ''[file attributes]'' on Darwin/Mac OS X will
cause the non-builtin ''macRoman'' encoding to load. ASCII-only
OSTypes will still work properly if ''macRoman'' is not available,
fallback to ''latin1'' in that case could also be added if deemed
necessary. However, the Tk Aqua port already relies on
''macRoman'' being present so in the most common usage pattern
''macRoman'' should be present and loaded anyway.
* The Mac OS 9 implementation of ''[file attributes -creator -type]''
currently returns the bogus 'Fldr' type & creator for directories,
this has been changed to return an error for consistency with the
Mac OS X implementation.
* Most of the implementation of the new Mac OS X specific features
has been added added at the end of ''unix/tclUnixFCmd.c'', it might
be cleaner to move this code to a separate file
''macosx/tclMacOSXFCmd.c'', but that would require several routines
in both ''unix/tclUnixFCmd.c'' and ''mac/tclMacOSXFCmd.c'' to be
made non-static. It's unclear whether this is an acceptable change
just for the sake of code separation/cleanliness.
~ Reference Implementation
SourceForge patch #626360
[[http://sourceforge.net/tracker/index.php?func=detail&aid=626360&group_id=10894&atid=310894]]
implements this TIP as a patch to the current HEAD.
The patch has been tested on the SF compile-farm on hosts:
[Alpha] Linux 2.4 (Debian 3.0): where ''chflags()'' and
''getattrlist()'' are not available and no ill effects ensue
(i.e. no new tests fail).
[x86] FreeBSD (4.7-RC): where ''chflags()'' is available and ''[file
attributes -readonly]'' can successfully be interrogated (but not
set due to permission issues at SourceForge). No new tests fail.
as well as on Mac OS 9, X 10.1.5 and X 10.2.1, where all the new
functionality is available and no new tests fail.
~ Copyright
This document has been placed in the public domain.

# TIP 118: Enhance [file attributes] and [file copy] on Mac OS X & BSD
Author: Daniel A. Steffen <das@users.sourceforge.net>
State: Final
Type: Project
Vote: Done
Created: 01-Nov-2002
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes a set of changes to _[file attributes]_ and
_[file copy]_ to make them function better on MacOS X and other
BSD-Unix systems.
# Summary of Proposed Changes
This TIP proposes five sets of changes to _[file attributes]_ and
_[file copy]_:
1. Add support for the Macintosh and Windows specific flag _[file
attributes -readonly]_ to Unixes that support the _chflags\(\)_
API, to allow the _user immutable_ flag to be accessed &
modified from Tcl.
2. Add support for the Macintosh specific flags _[file attributes
-readonly -creator -type -hidden]_ to Mac OS X via the POSIX
level API _getattrlist\(\)_.
3. Add a new flag _[file attributes -rsrclength]_ to Mac OS X and
Mac OS 9 that gives the length of the resource fork of a file; 0
is the only value that this attribute can be set to, which strips
the resource fork off a file.
4. Change _[file attributes]_ to return the list of attributes that
can be retrieved without error for a given file, instead of
aborting the whole command when any error occurs.
5. Enhance _[file copy]_ on Mac OS X \(more precisely, the native
file-system _Tcl\_FSCopyFile_\) to copy finder attributes \(i.e.
_-readonly -creator -type -hidden_\) and resource forks
transparently.
# Rationale
There is currently no way to access and modify HFS file-system metadata
from Tcl on Mac OS X whereas Tcl on Mac OS 9 \(or Classic\) on the same
Macintosh has that capability. Worse, _[file copy]_ \(and
potentially even _[file rename]_ if it results in a copy\) on Mac OS
X can be a destructive operation at present if it operates on a file
that has essential data in its resource fork or its HFS metadata.
This again in contrast to the same operations in Tcl on Mac OS 9 where
this information is preserved. This TIP seeks to rectify these
asymmetries in order to better hide such file-system related platform
specificities from the scripter.
# Details
Additional information & examples:
1. Unix versions that support _chflags\(\)_ include BSD >= 4.4 and
Darwin/Mac OS X \(where _user immutable_ is the flag
corresponding to the file locked state on the HFS file-system,
which is what _[file attributes -readonly]_ controls on Mac OS
9\).
2. The use of _getattrlist\(\)_ does not require linking with Carbon
and thus allows access to HFS file-system metadata from Tcl on pure
open-source Darwin systems \(which is something no other scripting
language can claim at present\).
3. The new attribute _-rsrclength_ is useful to check whether a
file has a resource fork and to calculate total file size on Mac
OS 9 and X \(note that _[file size]_ returns the size of the data
fork only\). Stripping a file's resource fork \(by setting
_-rsrclength_ to 0\) is a common operation on Mac OS when dealing
with files that are destined for other platforms. This is a
feature that has been requested several times and given that it
ties in well with the implementation of the other new attributes
it comes at essentially no additional cost.
% file attributes test
-group admin -owner steffen -permissions 00644 -readonly 0 -creator Doug -type RSRC -hidden 0 -rsrclength 314
% file attributes test -rsrclength 5
setting nonzero rsrclength not supported
% file attributes test -rsrclength 0
% file attributes test
-group admin -owner steffen -permissions 00644 -readonly 0 -creator Doug -type RSRC -hidden 0 -rsrclength 0
4. On Mac OS X, trying to retrieve the new attributes _-creator
-type -rsrclength_ fails on non-regular files & directories \(and
on any file located on a non-HFS file-system that doesn't support
_getattrlist\(\)_\). Returning only the list of attributes that
are valid seems like much more sensible behaviour in this case
than failing with an error and not returning anything. In the
case where no valid attributes can be retrieved at all, the error
returned by the last attempt is passed upstream, to preserve
existing error handling. This proposed change in behaviour of
_[file attributes]_ seems necessary to allow the command to
continue to work in a consistent way on all inputs and on all
platforms; it should not impact existing code since for current
attributes, failure to retrieve any one attribute is equivalent to
failure to retrieve all attributes.
% close [open test w]
% file attributes test
-group admin -owner steffen -permissions 00644 -readonly 0 -creator {} -type {} -hidden 0 -rsrclength 0
% file delete test
% file mkdir test
% file attributes test
-group admin -owner steffen -permissions 040755 -readonly 0 -hidden 0
% file delete test
5. Unlike the Finder and other HFS aware tools on Mac OS X, Tcl
currently ignores HFS metadata and the resource fork, which will
undoubtedly surprise scripters unpleasantly. _[file copy]_
should hide such platform specificities and copy a file in the
same way as the Finder:
% file attributes test
-group admin -owner steffen -permissions 00644 -readonly 0 -creator Doug -type RSRC -hidden 0 -rsrclength 314
% file copy test test1
% file attributes test1
-group admin -owner steffen -permissions 00644 -readonly 0 -creator Doug -type RSRC -hidden 0 -rsrclength 314
% file delete test1
# Comments
Additional implementation details:
* To support the new attributes _-creator -type_, routines to
convert from numerical OSTypes \(_u\_int32\_t_\) to the usual four
character human readable format have been adapted from
_mac/tclMacResource.c_; the new versions accept/return strings of
length 0-4 unlike the originals that only dealt with length 4.
This is important because creator/type 0 \(i.e. _-creator \{\} -type
\{\}_\) is common on Mac OS X. The Mac OS 9 implementation of the
OSType string representation code has been modified accordingly by
adding support for strings of length 0-4 and missing
_UtfToExternal/ExternalToUtf_ conversions.
* _macRoman_ is the encoding used for the string representation of
OSTypes, for consistency with Tcl on Mac OS 9 as well as with
common Mac OS X tools such as Resorcerer & SuperGetInfo that all
use _macRoman_ to display creator/type codes; this encoding is
probably what most people would expect. It's unfortunate that this
means that use of _[file attributes]_ on Darwin/Mac OS X will
cause the non-builtin _macRoman_ encoding to load. ASCII-only
OSTypes will still work properly if _macRoman_ is not available,
fallback to _latin1_ in that case could also be added if deemed
necessary. However, the Tk Aqua port already relies on
_macRoman_ being present so in the most common usage pattern
_macRoman_ should be present and loaded anyway.
* The Mac OS 9 implementation of _[file attributes -creator -type]_
currently returns the bogus 'Fldr' type & creator for directories,
this has been changed to return an error for consistency with the
Mac OS X implementation.
* Most of the implementation of the new Mac OS X specific features
has been added added at the end of _unix/tclUnixFCmd.c_, it might
be cleaner to move this code to a separate file
_macosx/tclMacOSXFCmd.c_, but that would require several routines
in both _unix/tclUnixFCmd.c_ and _mac/tclMacOSXFCmd.c_ to be
made non-static. It's unclear whether this is an acceptable change
just for the sake of code separation/cleanliness.
# Reference Implementation
SourceForge patch \#626360
<http://sourceforge.net/tracker/index.php?func=detail&aid=626360&group_id=10894&atid=310894>
implements this TIP as a patch to the current HEAD.
The patch has been tested on the SF compile-farm on hosts:
[Alpha] Linux 2.4 \(Debian 3.0\): where _chflags\(\)_ and
_getattrlist\(\)_ are not available and no ill effects ensue
\(i.e. no new tests fail\).
[x86] FreeBSD \(4.7-RC\): where _chflags\(\)_ is available and _[file
attributes -readonly]_ can successfully be interrogated \(but not
set due to permission issues at SourceForge\). No new tests fail.
as well as on Mac OS 9, X 10.1.5 and X 10.2.1, where all the new
functionality is available and no new tests fail.
# Copyright
This document has been placed in the public domain.

TIP: 119Title: Angled Text on a Canvas
Version: $Revision: 1.8 $
Author: Simon Geard <simon.geard@ntlworld.com>
Author: Donal K. Fellows <donal.k.fellows@manchester.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 18-Nov-2002
Post-History:
Tcl-Version: 8.6
~ Abstract
The current text object on a canvas does not support the creation of
text strings at an arbitrary angle. For some applications this is
limitation is sufficiently serious to disqualify Tk from use. This
TIP removes this restriction.
~ Rationale
Using a Tk canvas to display e.g. an engineering drawing from a CAD
application is limited - such applications will output angled text and
there is simply no way of displaying such text on a canvas.
Another consideration is the competition. Both Qt and Java canvas
objects support angled text on their canvas object - most new users
would I believe be surprised to discover that this functionality is not
supported.
Due to the internal architecture of Tk a side effect of this work
would be to enable buttons (and possibly other widgets) to have their
text written at an angle as well. This TIP does not expose that
functionality.
~ Implementation
At the command level there would be an extra attribute, ''-angle'',
added to the canvas's text object:
|<canvas> create text <x> <y> -text <str> ?-angle <angle>? ...
where the angle is specified in degrees. Manipulation of the angle
would be done in the usual way through the ''itemconfigure'' and
''itemcget'' subcommands:
|<canvas> itemconfigure <tag> -angle <angle>
Note that PostScript naturally supports rotation of text.
~ Reference Implementation
I have done an initial assessment of the amount of work required and
have and have created a patch to implement it in Tk 8.4.5 . This
initial work was fairly straightforward but the internal housekeeping
work maintaining and interacting with an angled box will be more
substantial. The patch (uploaded to SourceForge
[http://sf.net/tracker/?func=detail&aid=1611359&group_id=12997&atid=312997]) is
for X only. It doesn't use any X extensions and should work for X11R3
and above. I don't know how it would be done on non-X systems but
hopefully they'll (Win & MacOS at least) provide native tools to make
their implementation straightforward as well.
~~ Implementation Notes
The implementation was tricky to develop, because it was wholly different on
each rendering platform (X11 is two, depending on which font rendering engine
is used). This delayed implementation of this TIP for a whole release.
Classic X11: Uses a general monochrome pixmap rotation engine.
X11 + Xft: Generates rotated versions of the fonts by installing a rotation
matrix, caching them in the subfont management engine.
Win32: Generates rotated versions of the fonts using the built-in font
rotation suport, caching them in the subfont management engine.
MacOS X: Applies the rotation to the Current Transformation Matrix
immediately before rendering.
~ Copyright
This document has been placed in the public domain.

# TIP 119: Angled Text on a Canvas
Author: Simon Geard <simon.geard@ntlworld.com>
Author: Donal K. Fellows <donal.k.fellows@manchester.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 18-Nov-2002
Post-History:
Tcl-Version: 8.6
-----# Abstract
The current text object on a canvas does not support the creation of
text strings at an arbitrary angle. For some applications this is
limitation is sufficiently serious to disqualify Tk from use. This
TIP removes this restriction.
# Rationale
Using a Tk canvas to display e.g. an engineering drawing from a CAD
application is limited - such applications will output angled text and
there is simply no way of displaying such text on a canvas.
Another consideration is the competition. Both Qt and Java canvas
objects support angled text on their canvas object - most new users
would I believe be surprised to discover that this functionality is not
supported.
Due to the internal architecture of Tk a side effect of this work
would be to enable buttons \(and possibly other widgets\) to have their
text written at an angle as well. This TIP does not expose that
functionality.
# Implementation
At the command level there would be an extra attribute, _-angle_,
added to the canvas's text object:
<canvas> create text <x> <y> -text <str> ?-angle <angle>? ...
where the angle is specified in degrees. Manipulation of the angle
would be done in the usual way through the _itemconfigure_ and
_itemcget_ subcommands:
<canvas> itemconfigure <tag> -angle <angle>
Note that PostScript naturally supports rotation of text.
# Reference Implementation
I have done an initial assessment of the amount of work required and
have and have created a patch to implement it in Tk 8.4.5 . This
initial work was fairly straightforward but the internal housekeeping
work maintaining and interacting with an angled box will be more
substantial. The patch \(uploaded to SourceForge
<http://sf.net/tracker/?func=detail&aid=1611359&group_id=12997&atid=312997> \) is
for X only. It doesn't use any X extensions and should work for X11R3
and above. I don't know how it would be done on non-X systems but
hopefully they'll \(Win & MacOS at least\) provide native tools to make
their implementation straightforward as well.
## Implementation Notes
The implementation was tricky to develop, because it was wholly different on
each rendering platform \(X11 is two, depending on which font rendering engine
is used\). This delayed implementation of this TIP for a whole release.
Classic X11: Uses a general monochrome pixmap rotation engine.
X11 \+ Xft: Generates rotated versions of the fonts by installing a rotation
matrix, caching them in the subfont management engine.
Win32: Generates rotated versions of the fonts using the built-in font
rotation suport, caching them in the subfont management engine.
MacOS X: Applies the rotation to the Current Transformation Matrix
immediately before rendering.
# Copyright
This document has been placed in the public domain.

TIP: 12Title: The "Batteries Included" Distribution
Version: $Revision: 1.3 $
Author: George A. Howlett <gah@siliconmetrics.com>
Author: Larry W. Virden <lvirden@yahoo.com>
State: Draft
Type: Informative
Vote: Pending
Created: 15-Sep-2000
Post-History:
Discussions-To: news:comp.lang.tcl
~ Abstract
This document describes a comprehensive Tcl/Tk distribution. Its
primary purpose is to create a standard source tree that includes Tcl,
Tk, and extensions so that they can be built and installed in an
simple and easy manner.
~ Introduction
One of the most enduring complaints about Tcl/Tk is that it lacks
features, especially when compared to Perl, Python, or Java. We
patiently explain that some particular feature is available in
extension "XYZ" only to hear how hard it is to build and install
extensions.
Frank Stajano ("The SMS server, or why I switched from Tcl to Python")
describes the problem succinctly.
> "But if I had to put the finger on the single most important reason
that has me now working in Python rather than in Tcl/[[incr Tcl]]
it would not be a language issue but a library issue. I prefer
Python because its standard library is a gold mine. Sure, for
anything I want to do there's bound to be an extension available in
the Tcl code repository on the FTP site. Now I just have to find
it, fetch it, recompile the interpreter with it (Oh wait - this may
mean getting and installing a C compiler for this system. Will the
GNU one compile the windowing stuff properly or do I need to get
VC++, or Borland? Who wants to have some fun discovering where
another IDE has hidden the useful compiler flags this week?), hope
that it won't clash with other extensions I've had to install, hope
that it will not require a different version of the interpreter
from the one I am running, and so on. Python supports the same C
extension mechanism as Tcl - but the practical difference is that
the stuff I want is, most of the time, already included and shipped
in the standard distribution of the language!"
................................................................................
that words "core" and "extension" disappear from our Tcl vocabularies.
We've lived their artifical distinctions that are useful only to core
developers and extension writers. It's skewed our thinking about
relationship between Tcl and its parts. After all, application
writers first care about whether a feature or capability is available,
not how it's structured under the hood.
~ The "Batteries Included" Distribution.
Let's start with a very modest example. Let's imagine that the
"Batteries Included" distribution is nothing more than an archive file of
the source code for Tcl/Tk and several extensions.
| Unix Windows Mac
| ---- ------- ---
| Tcl 8.3 x x x
| Tk 8.3 x x x
| [incr Tcl] x x x
| expect x ?
| TclX x
| BLT x x
| Trf
| Html widget
| XML
| ...lots more...
Tcl, Tk, and the packages are configured such that they can be built
and installed just from a top level directory (not individually).
Someone can download and try out all sorts of new features without
repeating the same "configure", "make", "make install" sequences.
With this simple tar file, the following benefits are automatically
generated:
* It provides a simple way for users to try out extensions. Users
................................................................................
features being universally available. Your program can again be
just a Tcl script, not an array of packages that everyone needs to
download and install.
* It's better for extension writers. Configuration is simpler,
since you know where all the sources and the compiler-specific
information will reside. You don't need to search for
''tclConfig.sh'' or ''tkConfig.sh'' files.
* It's better for Tcl/Tk distribution builders. This includes both
the Linux distributors and company sysadmins that build Tcl/Tk.
They don't have to fear installing extensions because of version
dependencies.
> Let's give Redhat and SuSE a good reason to move off of version
8.0. One the big advantages of Linux over (let's say) Solaris is
that each new Redhat or SuSE distribution comes with updated
versions of utilities already built.
* It's better for the core developers. Extension writers will
willing the adopt changes in exchange for the wider distribution.
The core team will in turn gain better understanding of the
burdens of extension writers.
* It's better for Tcl library writers. With [incr Tcl], we now have
a basis for a real, extensible Tcl-code library. Library code
rely on a full set of extensions being available.
~ Rationale
We want to create an open door procedure that makes it easy for
contributors to add new features and commands to Tcl and Tk. By
creating a framework for extensions to be built and distributed, the
"Batteries Included" distribution will provide a path for great new
features to quickly become available to the Tcl community.
................................................................................
If the "Batteries Included" distribution is to become successful, it
must be a cooperative effort between Tcl core developers, extension
writers, and the Tcl user community. For example, we need the help of
extension writers to adopt the new configuration scheme and directory
structure.
~ Particulars
We can stage the project with small milestones while still focusing on
longer range goals. For example, the first phase can be as simple as
creating a tar file. It will start to address questions that were
raised by TEA. For example, how do we manage documentation?
The biggest reason why this proposal will succeed is the incredible
talent in the Tcl community. We can leverage the skills and
experiences of the foremost experts on the core, extensions, and
applications.
~ Tcl/Tk Version.
The distribution will be based on 8.3.2 (or 8.3.3 when it is
released). While there's no assurance when 8.4 will be released and
in what state, we also want to place a premium on stable, robust
extensions, that have been thoroughly tested. Most extensions will be
unlikely to have been tested against the 8.4 alphas.
~ Phase 1.
* Identify extensions.
> What extensions should be included in the near term? We need
extension authors that are willing to work with us to build a
directory framework, change configuration files, etc. Extensions
do not need to work on all platforms. For example, there is a
wealth of Windows-based extensions that should be included in
a Windows specific build.
> What are the minimum requirements for extensions in the short term?
Manual pages, html, tests, demos all would be nice. We need to
temper this with what's practical. This is a learning process. We
can adjust requirements in future phases.
* Determine build and install directory structures.
> We need to make this work with more that one release installed.
Don't suppose that there only one version will ever be used.
* Setup CVS archives.
* Create configuration files.
> This will require negotiation with extension writers. We want
their buy-in so they will maintain the changes.
> There may be more than one form of configuration required. One
subtle but important issue is that extensions must be able to be
configured without Tcl or Tk libraries already existing. This is a
"trusted" configure. The extension must trust that the library
will exist. Right now, most extensions work from "untrusted"
configurations.
* Test builds on multiple platforms.
> For now, the Windows and Mac build files can be hand-generated. It
may be too hard to create a seamless build environment. We're not
trying to satisfy every Windows/Mac developer here. We can focus
on creating pre-built binary distributions for these platforms.
* Create self-installing executables for Windows and the Mac.
> If we want, we can provide Linux, Solaris, etc. binaries by
reviving Michael McLennan's Tclish installer.
~ Phase 2.
* Handle documentation issues.
> Generate platform specific doc with Richard Hipp's XML code.
* Establish Tcl code library.
* Identify more extensions.
* Determine the release schedule for "batteries included" distribution.
> How often do you release a new version? It must be more frequent
than Tcl/Tk. We can start by planning for quarterly releases and
then adding more frequent releases if necessary.
* Determine what core changes (if any) are needed for the distribution.
* Start looking at network-based updates.
* Start looking at selective builds. Allow builders to compile/install
subsets of the distribution.
* Push on Redhat, SuSE, etc. to pick up distribution.
~ Phase 3.
* Network-based installs.
* Selective installations/builds.
* Include applications tree.
* Identify more extensions.
The last phases are sketchy. Feel free to add to this list, further
breaking down goals into subtasks.
~ Open Issues
* Windows and MacIntosh sources.
> Given the dearth of configuration tools for these platforms, it's
likely that only binary installations will be available for the
near term.
* Documentation
> Overlap in command and widget names can be neatly handled by
namespaces. Need to consider how to handle manual pages.
~ More Information
If anyone has interest to participate or would like to add comments to
the "Batteries Included" proposal, please send mail to George Howlett
<gah@siliconmetrics.com>.
~ Copyright
This document has been placed in the public domain.
~ See Also
[4] by Brent Welch <welch@acm.org>.

# TIP 12: The "Batteries Included" Distribution
Author: George A. Howlett <gah@siliconmetrics.com>
Author: Larry W. Virden <lvirden@yahoo.com>
State: Draft
Type: Informative
Vote: Pending
Created: 15-Sep-2000
Post-History:
Discussions-To: news:comp.lang.tcl
-----# Abstract
This document describes a comprehensive Tcl/Tk distribution. Its
primary purpose is to create a standard source tree that includes Tcl,
Tk, and extensions so that they can be built and installed in an
simple and easy manner.
# Introduction
One of the most enduring complaints about Tcl/Tk is that it lacks
features, especially when compared to Perl, Python, or Java. We
patiently explain that some particular feature is available in
extension "XYZ" only to hear how hard it is to build and install
extensions.
Frank Stajano \("The SMS server, or why I switched from Tcl to Python"\)
describes the problem succinctly.
> "But if I had to put the finger on the single most important reason
that has me now working in Python rather than in Tcl/[incr Tcl]
it would not be a language issue but a library issue. I prefer
Python because its standard library is a gold mine. Sure, for
anything I want to do there's bound to be an extension available in
the Tcl code repository on the FTP site. Now I just have to find
it, fetch it, recompile the interpreter with it \(Oh wait - this may
mean getting and installing a C compiler for this system. Will the
GNU one compile the windowing stuff properly or do I need to get
VC\+\+, or Borland? Who wants to have some fun discovering where
another IDE has hidden the useful compiler flags this week?\), hope
that it won't clash with other extensions I've had to install, hope
that it will not require a different version of the interpreter
from the one I am running, and so on. Python supports the same C
extension mechanism as Tcl - but the practical difference is that
the stuff I want is, most of the time, already included and shipped
in the standard distribution of the language!"
................................................................................
that words "core" and "extension" disappear from our Tcl vocabularies.
We've lived their artifical distinctions that are useful only to core
developers and extension writers. It's skewed our thinking about
relationship between Tcl and its parts. After all, application
writers first care about whether a feature or capability is available,
not how it's structured under the hood.
# The "Batteries Included" Distribution.
Let's start with a very modest example. Let's imagine that the
"Batteries Included" distribution is nothing more than an archive file of
the source code for Tcl/Tk and several extensions.
Unix Windows Mac
---- ------- ---
Tcl 8.3 x x x
Tk 8.3 x x x
[incr Tcl] x x x
expect x ?
TclX x
BLT x x
Trf
Html widget
XML
...lots more...
Tcl, Tk, and the packages are configured such that they can be built
and installed just from a top level directory \(not individually\).
Someone can download and try out all sorts of new features without
repeating the same "configure", "make", "make install" sequences.
With this simple tar file, the following benefits are automatically
generated:
* It provides a simple way for users to try out extensions. Users
................................................................................
features being universally available. Your program can again be
just a Tcl script, not an array of packages that everyone needs to
download and install.
* It's better for extension writers. Configuration is simpler,
since you know where all the sources and the compiler-specific
information will reside. You don't need to search for
_tclConfig.sh_ or _tkConfig.sh_ files.
* It's better for Tcl/Tk distribution builders. This includes both
the Linux distributors and company sysadmins that build Tcl/Tk.
They don't have to fear installing extensions because of version
dependencies.
> Let's give Redhat and SuSE a good reason to move off of version
8.0. One the big advantages of Linux over \(let's say\) Solaris is
that each new Redhat or SuSE distribution comes with updated
versions of utilities already built.
* It's better for the core developers. Extension writers will
willing the adopt changes in exchange for the wider distribution.
The core team will in turn gain better understanding of the
burdens of extension writers.
* It's better for Tcl library writers. With [incr Tcl], we now have
a basis for a real, extensible Tcl-code library. Library code
rely on a full set of extensions being available.
# Rationale
We want to create an open door procedure that makes it easy for
contributors to add new features and commands to Tcl and Tk. By
creating a framework for extensions to be built and distributed, the
"Batteries Included" distribution will provide a path for great new
features to quickly become available to the Tcl community.
................................................................................
If the "Batteries Included" distribution is to become successful, it
must be a cooperative effort between Tcl core developers, extension
writers, and the Tcl user community. For example, we need the help of
extension writers to adopt the new configuration scheme and directory
structure.
# Particulars
We can stage the project with small milestones while still focusing on
longer range goals. For example, the first phase can be as simple as
creating a tar file. It will start to address questions that were
raised by TEA. For example, how do we manage documentation?
The biggest reason why this proposal will succeed is the incredible
talent in the Tcl community. We can leverage the skills and
experiences of the foremost experts on the core, extensions, and
applications.
# Tcl/Tk Version.
The distribution will be based on 8.3.2 \(or 8.3.3 when it is
released\). While there's no assurance when 8.4 will be released and
in what state, we also want to place a premium on stable, robust
extensions, that have been thoroughly tested. Most extensions will be
unlikely to have been tested against the 8.4 alphas.
# Phase 1.
* Identify extensions.
> What extensions should be included in the near term? We need
extension authors that are willing to work with us to build a
directory framework, change configuration files, etc. Extensions
do not need to work on all platforms. For example, there is a
wealth of Windows-based extensions that should be included in
a Windows specific build.
> What are the minimum requirements for extensions in the short term?
Manual pages, html, tests, demos all would be nice. We need to
temper this with what's practical. This is a learning process. We
can adjust requirements in future phases.
* Determine build and install directory structures.
> We need to make this work with more that one release installed.
Don't suppose that there only one version will ever be used.
* Setup CVS archives.
* Create configuration files.
> This will require negotiation with extension writers. We want
their buy-in so they will maintain the changes.
> There may be more than one form of configuration required. One
subtle but important issue is that extensions must be able to be
configured without Tcl or Tk libraries already existing. This is a
"trusted" configure. The extension must trust that the library
will exist. Right now, most extensions work from "untrusted"
configurations.
* Test builds on multiple platforms.
> For now, the Windows and Mac build files can be hand-generated. It
may be too hard to create a seamless build environment. We're not
trying to satisfy every Windows/Mac developer here. We can focus
on creating pre-built binary distributions for these platforms.
* Create self-installing executables for Windows and the Mac.
> If we want, we can provide Linux, Solaris, etc. binaries by
reviving Michael McLennan's Tclish installer.
# Phase 2.
* Handle documentation issues.
> Generate platform specific doc with Richard Hipp's XML code.
* Establish Tcl code library.
* Identify more extensions.
* Determine the release schedule for "batteries included" distribution.
> How often do you release a new version? It must be more frequent
than Tcl/Tk. We can start by planning for quarterly releases and
then adding more frequent releases if necessary.
* Determine what core changes \(if any\) are needed for the distribution.
* Start looking at network-based updates.
* Start looking at selective builds. Allow builders to compile/install
subsets of the distribution.
* Push on Redhat, SuSE, etc. to pick up distribution.
# Phase 3.
* Network-based installs.
* Selective installations/builds.
* Include applications tree.
* Identify more extensions.
The last phases are sketchy. Feel free to add to this list, further
breaking down goals into subtasks.
# Open Issues
* Windows and MacIntosh sources.
> Given the dearth of configuration tools for these platforms, it's
likely that only binary installations will be available for the
near term.
* Documentation
> Overlap in command and widget names can be neatly handled by
namespaces. Need to consider how to handle manual pages.
# More Information
If anyone has interest to participate or would like to add comments to
the "Batteries Included" proposal, please send mail to George Howlett
<gah@siliconmetrics.com>.
# Copyright
This document has been placed in the public domain.
# See Also
[[4]](4.md) by Brent Welch <welch@acm.org>.

TIP: 120Title: Restricted DDE Services
Version: $Revision: 1.5 $
Author: Pat Thoyts <patthoyts@users.sourceforge.net>
State: Final
Type: Project
Vote: Done
Created: 04-Dec-2002
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP will enhance the DDE package for use with safe interpreters
and allow programmer control of the commands exposed by the DDE
service.
~ Rationale
By default the Tcl DDE service exposes all of the commands and
variables available in the interpreter to all DDE clients. This is
not always desirable. One solution might be to load the package into
a safe slave interpreter and use [[interp alias]] to expose the
required commands. Unfortunately the package doesn't support loading
into safe interpreters.
~ Proposed Changes
Firstly, this TIP proposes a ''-handler'' option to the [[dde
servername]] sub-command. The argument for this option should be the
name of a procedure that will authenticate and evaluate DDE requests.
Secondly, the DDE package should be enhanced to be capable of
providing a service within a safe interpreter.
~ New Option
The new syntax will be ''dde servername ?-handler command?
servername''. To permit introspection we will accept ''dde servername
-handler'' which will return the handler name (if any). If a
servername must be defined using an initial hyphen then the standard
'--' separator can be used.
The dde request is appended to the handler command (which may be a list)
and then evaluated in the global context. This ensures that all unsafe
elements will not be evaluated before the handler code has a chance to
examine them. So
| proc handler {request} {
| if {[string match "info *" $request} {
| uplevel #0 [lindex $request 0] [lrange $request 1 end]
| } else
| return -code error "permission denied"
| }| }
The above handler will permit [[info vars]] but will fail when trying
[[info vars ; bad_proc]] with info complaining about the wrong number
of parameters. Passing a single string means that we can handle
standard dde styles of requests, for instance
'Open("c:\Program Files\prog.exe")' which would not usefully convert
into a list.
~ Safe DDE
The dde package should support loading within a safe interpreter but
with the following constraints.
* The dde command should be hidden. This means that the safe
interpreter may not call the command but a master interpreter call
this command within the context of the safe interpreter.
* Remote execution requests should be handled ''only'' by a defined
handler procedure. The normal default is to evaluate a remote
execution request in the global namespace. I propose that when
operating in a safe interpreter that the request be denied unless a
handler is defined. The programmer then has the ability to
authenticate the request before it is evaluated.
* Remote variable reads should be denied. Rather that add in another
handler - the XTYP_REQUEST service command should be denied for
safe interpreters. It is trivial to use [[dde eval Remote set
$varname]] to read the value of a variable.
~ Reference Implementation
See the SourceForge Feature Request at
https://sourceforge.net/tracker/index.php?func=detail&aid=649859&group_id=10894&atid=310894~ Example
| # Provide a handler that only allows the [info] command
| # Note: This runs in the master interp.
| proc restricted_handler {request} {
| if {[string match "info *" $request} {
| uplevel #0 [lindex $request 0] [lrange $request 1 end]
| } else
| return -code error "permission denied"
| }| }|| # Create a safe slave interpreter and expose as a DDE service.
| safe::interpCreate slave
| slave invokehidden load tcldde12d.dll Dde
| slave invokehidden dde servername -handler dde_cmd SafeSlave
| interp alias slave dde_cmd {} restricted_handler
|| # If testing in tclsh...
| set ::_waiting 0 ; after 20000 {set ::_waiting 1} ; vwait ::_waiting
~ Consequences
There should be no change to current users of this package unless they
are using a server name beginning with a hyphen. In this case they
will need to insert '--' before the server name.
~ Copyright
This document is hereby placed in the public domain.

# TIP 120: Restricted DDE Services
Author: Pat Thoyts <patthoyts@users.sourceforge.net>
State: Final
Type: Project
Vote: Done
Created: 04-Dec-2002
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP will enhance the DDE package for use with safe interpreters
and allow programmer control of the commands exposed by the DDE
service.
# Rationale
By default the Tcl DDE service exposes all of the commands and
variables available in the interpreter to all DDE clients. This is
not always desirable. One solution might be to load the package into
a safe slave interpreter and use [interp alias] to expose the
required commands. Unfortunately the package doesn't support loading
into safe interpreters.
# Proposed Changes
Firstly, this TIP proposes a _-handler_ option to the [dde
servername] sub-command. The argument for this option should be the
name of a procedure that will authenticate and evaluate DDE requests.
Secondly, the DDE package should be enhanced to be capable of
providing a service within a safe interpreter.
# New Option
The new syntax will be _dde servername ?-handler command?
servername_. To permit introspection we will accept _dde servername
-handler_ which will return the handler name \(if any\). If a
servername must be defined using an initial hyphen then the standard
'--' separator can be used.
The dde request is appended to the handler command \(which may be a list\)
and then evaluated in the global context. This ensures that all unsafe
elements will not be evaluated before the handler code has a chance to
examine them. So
proc handler {request} {
if {[string match "info *" $request} {
uplevel #0 [lindex $request 0] [lrange $request 1 end]
} else
return -code error "permission denied"
} }
The above handler will permit [info vars] but will fail when trying
[info vars ; bad_proc] with info complaining about the wrong number
of parameters. Passing a single string means that we can handle
standard dde styles of requests, for instance
'Open\("c:\\Program Files\\prog.exe"\)' which would not usefully convert
into a list.
# Safe DDE
The dde package should support loading within a safe interpreter but
with the following constraints.
* The dde command should be hidden. This means that the safe
interpreter may not call the command but a master interpreter call
this command within the context of the safe interpreter.
* Remote execution requests should be handled _only_ by a defined
handler procedure. The normal default is to evaluate a remote
execution request in the global namespace. I propose that when
operating in a safe interpreter that the request be denied unless a
handler is defined. The programmer then has the ability to
authenticate the request before it is evaluated.
* Remote variable reads should be denied. Rather that add in another
handler - the XTYP\_REQUEST service command should be denied for
safe interpreters. It is trivial to use [dde eval Remote set
$varname] to read the value of a variable.
# Reference Implementation
See the SourceForge Feature Request at
<https://sourceforge.net/tracker/index.php?func=detail&aid=649859&group\_id=10894&atid=310894># Example
# Provide a handler that only allows the [info] command
# Note: This runs in the master interp.
proc restricted_handler {request} {
if {[string match "info *" $request} {
uplevel #0 [lindex $request 0] [lrange $request 1 end]
} else
return -code error "permission denied"
} } # Create a safe slave interpreter and expose as a DDE service.
safe::interpCreate slave
slave invokehidden load tcldde12d.dll Dde
slave invokehidden dde servername -handler dde_cmd SafeSlave
interp alias slave dde_cmd {} restricted_handler
# If testing in tclsh...
set ::_waiting 0 ; after 20000 {set ::_waiting 1} ; vwait ::_waiting
# Consequences
There should be no change to current users of this package unless they
are using a server name beginning with a hyphen. In this case they
will need to insert '--' before the server name.
# Copyright
This document is hereby placed in the public domain.

TIP: 121Title: Controlled Application Shutdown via Tcl_Exit
Version: $Revision: 1.8 $
Author: Joe Mistachkin <joe@mistachkin.com>
State: Final
Type: Project
Vote: Done
Created: 05-Dec-2002
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP will allow all applications to perform a controlled shutdown
(or do nothing) in the event that ''Tcl_Exit()'' is called.~ Rationale
For most applications written in C that use Tcl, calling the runtime
''exit()'' function is normally a reasonable way to shutdown the
application. Unfortunately, this is not always the case.
Applications written in other languages (and very complex C
applications) may have very specific application shutdown
requirements. This is especially true in multi-threaded
environments. The problem is further compounded by extensions that
use ''Tcl_Exit''.~ Versus Exit Handlers
There are distinct advantages to using this method instead of using
normal exit handlers in some cases. The normal exit handler cannot
defer (or in an emergency, prevent) the application shutdown because
''Tcl_Finalize()'' has already been called. From the perspective of
the exit handler, we have no way of knowing what other exit handlers
have been called and/or their subsystems destroyed. In addition, even
if it could somehow cause ''Tcl_Finalize()'' to defer the application
shutdown, some or all of the other exit handlers may have already
been executed, which would leave the application in an inconsistent
state. This relatively simple change allows the programmer to get
control very early during the application shutdown process.
~ Proposed Changes
First, this TIP proposes a new Tcl API function called
''Tcl_SetExitProc()'' or something similar. This function
accepts a ''Tcl_ExitProc'' pointer and returns a ''Tcl_ExitProc''
pointer. The return value is the old exit proc pointer.
Second, the ''Tcl_Exit()'' Tcl API function would be modified to
allow for the handler to be called.
Third, the documentation for ''Tcl_Exit()'' would be updated to
include ''Tcl_SetExitProc()'' and a warning that any custom exit proc
should NOT return.
~ Tcl_SetExitProc
|Tcl_ExitProc *
|Tcl_SetExitProc(proc)
| Tcl_ExitProc *proc; /* new exit handler for app or NULL */
|{| Tcl_ExitProc *prevExitProc; /* return prev exit handler to caller */
|| Tcl_MutexLock(&exitMutex);
| prevExitProc = appExitPtr; /* get old app exit ptr */
| appExitPtr = proc; /* set new app exit ptr */
| Tcl_MutexUnlock(&exitMutex);
|| return prevExitProc;
|}~ Tcl_Exit
|void
|Tcl_Exit(status)
| int status; /* Exit status for application; typically
| * 0 for normal return, 1 for error return. */
|{| Tcl_ExitProc *currentAppExitPtr;
|| Tcl_MutexLock(&exitMutex);
| currentAppExitPtr = appExitPtr;
| Tcl_MutexUnlock(&exitMutex);
|| if (currentAppExitPtr) {
| /***********************************************************/
| /* WARNING: This code SHOULD NOT return, as there is code */
| /* that depends on Tcl_Exit never returning. */
| /***********************************************************/
| currentAppExitPtr((ClientData) status);
| } else {
| Tcl_Finalize();
| TclpExit(status);
| }|| Tcl_Panic ("exitProc returned!");
|}~ Example
|void MyAppExitProc(ClientData clientData)
|{| /* #1. Perform application specific shutdown code... */
| /* #2. Wait for other threads to gracefully shutdown... */
|| exit(0);
|| return; /* We should never get here. */
|}||| ... sometime during application initialization ...
|| /* from this point on MyAppExitProc will handle Tcl_Exit requests */
| Tcl_SetExitProc(MyAppExitProc);
|| ... optionally, sometime later ...
|| /* from this point on the old (default) handling will be used */
| Tcl_SetExitProc(NULL);
~ Reference Implementation
A patch that implements everything suggested in this TIP is available
at the URL:
http://sourceforge.net/tracker/index.php?func=detail&aid=649313&group_id=10894&atid=310894~ Consequences
There is no change for current users of the ''Tcl_Exit()'' function,
including the [[exit]] command, if ''Tcl_SetExitProc()'' is never
called explicitly by the application.
~ Copyright
This document is hereby placed in the public domain.

# TIP 121: Controlled Application Shutdown via Tcl_Exit
Author: Joe Mistachkin <joe@mistachkin.com>
State: Final
Type: Project
Vote: Done
Created: 05-Dec-2002
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP will allow all applications to perform a controlled shutdown
\(or do nothing\) in the event that _Tcl\_Exit\(\)_ is called.# Rationale
For most applications written in C that use Tcl, calling the runtime
_exit\(\)_ function is normally a reasonable way to shutdown the
application. Unfortunately, this is not always the case.
Applications written in other languages \(and very complex C
applications\) may have very specific application shutdown
requirements. This is especially true in multi-threaded
environments. The problem is further compounded by extensions that
use _Tcl\_Exit_.# Versus Exit Handlers
There are distinct advantages to using this method instead of using
normal exit handlers in some cases. The normal exit handler cannot
defer \(or in an emergency, prevent\) the application shutdown because
_Tcl\_Finalize\(\)_ has already been called. From the perspective of
the exit handler, we have no way of knowing what other exit handlers
have been called and/or their subsystems destroyed. In addition, even
if it could somehow cause _Tcl\_Finalize\(\)_ to defer the application
shutdown, some or all of the other exit handlers may have already
been executed, which would leave the application in an inconsistent
state. This relatively simple change allows the programmer to get
control very early during the application shutdown process.
# Proposed Changes
First, this TIP proposes a new Tcl API function called
_Tcl\_SetExitProc\(\)_ or something similar. This function
accepts a _Tcl\_ExitProc_ pointer and returns a _Tcl\_ExitProc_
pointer. The return value is the old exit proc pointer.
Second, the _Tcl\_Exit\(\)_ Tcl API function would be modified to
allow for the handler to be called.
Third, the documentation for _Tcl\_Exit\(\)_ would be updated to
include _Tcl\_SetExitProc\(\)_ and a warning that any custom exit proc
should NOT return.
# Tcl\_SetExitProc
Tcl_ExitProc *
Tcl_SetExitProc(proc)
Tcl_ExitProc *proc; /* new exit handler for app or NULL */
{ Tcl_ExitProc *prevExitProc; /* return prev exit handler to caller */
Tcl_MutexLock(&exitMutex);
prevExitProc = appExitPtr; /* get old app exit ptr */
appExitPtr = proc; /* set new app exit ptr */
Tcl_MutexUnlock(&exitMutex);
return prevExitProc;
}# Tcl\_Exit
void
Tcl_Exit(status)
int status; /* Exit status for application; typically
* 0 for normal return, 1 for error return. */
{ Tcl_ExitProc *currentAppExitPtr;
Tcl_MutexLock(&exitMutex);
currentAppExitPtr = appExitPtr;
Tcl_MutexUnlock(&exitMutex);
if (currentAppExitPtr) {
/***********************************************************/
/* WARNING: This code SHOULD NOT return, as there is code */
/* that depends on Tcl_Exit never returning. */
/***********************************************************/
currentAppExitPtr((ClientData) status);
} else {
Tcl_Finalize();
TclpExit(status);
} Tcl_Panic ("exitProc returned!");
}# Example
void MyAppExitProc(ClientData clientData)
{ /* #1. Perform application specific shutdown code... */
/* #2. Wait for other threads to gracefully shutdown... */
exit(0);
return; /* We should never get here. */
} ... sometime during application initialization ...
/* from this point on MyAppExitProc will handle Tcl_Exit requests */
Tcl_SetExitProc(MyAppExitProc);
... optionally, sometime later ...
/* from this point on the old (default) handling will be used */
Tcl_SetExitProc(NULL);
# Reference Implementation
A patch that implements everything suggested in this TIP is available
at the URL:
<http://sourceforge.net/tracker/index.php?func=detail&aid=649313&group\_id=10894&atid=310894># Consequences
There is no change for current users of the _Tcl\_Exit\(\)_ function,
including the [exit] command, if _Tcl\_SetExitProc\(\)_ is never
called explicitly by the application.
# Copyright
This document is hereby placed in the public domain.

TIP: 122Title: Use tcl_{non,}wordchars Throughout Tcl/Tk
Version: $Revision: 1.9 $
Author: Martin Weber <ephaeton@gmx.net>
Author: Vince Darley <vincentdarley@users.sourceforge.net>
State: Rejected
Type: Project
Vote: Done
Created: 12-Dec-2002
Post-History:
Tcl-Version: 8.6
~ Abstract
This TIP shall bring flexible management of word and non-word chars to
Tcl, to be used throughout the Tcl realm in e.g. [[regexp]]'s \w \W,
Tk's [[textwidget]], [[string]] wordstart/wordend etc.
~Specification
Assignment to ''tcl_{non,}wordchars'' shall influence any place in Tcl
which decides whether something is a word character or not, including
detection of word boundaries in e.g. regular expressions, Tk's text
widget and so on.
For this there shall be no hard-coding of lists of values which are
word and non-word characters, and neither shall the language rely on
the language of implementation (i.e. C's ''is*()'' functions), as this
disallows dynamic changing of ''tcl_{non,}wordchars''.
Rather shall the value(s) of ''tcl_{non,}wordchars'' be used to
determine whether a given character is part of a word or not.
~Rationale
Currently in Tcl there are different hard-coded ways to decide whether
a certain character is a word character or a non word character.
Different hard-coded ways also imply that changes on one side might
not get over to the other side, so there soon are different hard-coded
ways which yield different hard-coded results. As a inference of it
being hard-coded, this also means that there is no way to change or
fix that potentially broken behavior. Having Tcl lookup the values of
those variables at runtime allows for the needed flexibility, both
when dealing with nonstandard demands and nonstandard character sets.
As an example of the breakage, you can assign a regular expression
to tcl_{,non}wordchars, and the double click binding in the textwidget
will regard that pattern when marking a "whole word". When you try
to ask the text widget to deliver the data under a certain coordinate
with the indices 'wortstart' and 'wordend', the value of
tcl_{non,}wordchars is not used though.
There may be a problem with the performance of the lookup, but on the
other hand are C's ''is*()'' functions also implemented via a table
lookup. An installation of a caching static character table could
guarantee the needed performance.
~Example of current word confusion
Tk's text widget uses "word" in several ways:
1. selection by word (double-click + drag),
2. movement by word ('insert wordstart'),
3. regexp searching with \m\M wordmatching.
4. line breaks when wrapping (-wrap word)
It is not at all clear from reading Tcl or Tk's documentation what the
behaviour of the above options will be. It turns out that:
1. after a convoluted call-chain, ends up calling
tcl_wordBreakAfter/Before which use tcl_wordchars and tcl_nonwordchars (which actually are defined differently on Windows vs Unix/MacOS!!).
2. uses 'isalnum(char)' or '_' to define a word (hard-coded in Tk's
tkTextIndex.c) (in Tk8.5a0 this has been fixed to use ''Tcl_UniCharIsWordChar'')
3. uses Tcl's regexp engine's definition of a word (this ought to be the same as that used in (2)).
4. Anything separated by white-space from something else, used with '-wrap word' to define line-wrapping in text widgets (and canvases).
It is quite likely that most of the above are different under some
circumstances or some platforms/locales, and certainly if the
user/developer wants to create a text widget with a different word
definition, they basically can't in any consistent way.
~Implementation
None yet.
~Copyright
This document is placed in the public domain.

# TIP 122: Use tcl_{non,}wordchars Throughout Tcl/Tk
Author: Martin Weber <ephaeton@gmx.net>
Author: Vince Darley <vincentdarley@users.sourceforge.net>
State: Rejected
Type: Project
Vote: Done
Created: 12-Dec-2002
Post-History:
Tcl-Version: 8.6
-----# Abstract
This TIP shall bring flexible management of word and non-word chars to
Tcl, to be used throughout the Tcl realm in e.g. [regexp]'s \\w \\W,
Tk's [textwidget], [string] wordstart/wordend etc.
# Specification
Assignment to _tcl\_\{non,\}wordchars_ shall influence any place in Tcl
which decides whether something is a word character or not, including
detection of word boundaries in e.g. regular expressions, Tk's text
widget and so on.
For this there shall be no hard-coding of lists of values which are
word and non-word characters, and neither shall the language rely on
the language of implementation \(i.e. C's _is\*\(\)_ functions\), as this
disallows dynamic changing of _tcl\_\{non,\}wordchars_.
Rather shall the value\(s\) of _tcl\_\{non,\}wordchars_ be used to
determine whether a given character is part of a word or not.
# Rationale
Currently in Tcl there are different hard-coded ways to decide whether
a certain character is a word character or a non word character.
Different hard-coded ways also imply that changes on one side might
not get over to the other side, so there soon are different hard-coded
ways which yield different hard-coded results. As a inference of it
being hard-coded, this also means that there is no way to change or
fix that potentially broken behavior. Having Tcl lookup the values of
those variables at runtime allows for the needed flexibility, both
when dealing with nonstandard demands and nonstandard character sets.
As an example of the breakage, you can assign a regular expression
to tcl\_\{,non\}wordchars, and the double click binding in the textwidget
will regard that pattern when marking a "whole word". When you try
to ask the text widget to deliver the data under a certain coordinate
with the indices 'wortstart' and 'wordend', the value of
tcl\_\{non,\}wordchars is not used though.
There may be a problem with the performance of the lookup, but on the
other hand are C's _is\*\(\)_ functions also implemented via a table
lookup. An installation of a caching static character table could
guarantee the needed performance.
# Example of current word confusion
Tk's text widget uses "word" in several ways:
1. selection by word \(double-click \+ drag\),
2. movement by word \('insert wordstart'\),
3. regexp searching with \\m\\M wordmatching.
4. line breaks when wrapping \(-wrap word\)
It is not at all clear from reading Tcl or Tk's documentation what the
behaviour of the above options will be. It turns out that:
1. after a convoluted call-chain, ends up calling
tcl\_wordBreakAfter/Before which use tcl\_wordchars and tcl\_nonwordchars \(which actually are defined differently on Windows vs Unix/MacOS!!\).
2. uses 'isalnum\(char\)' or '\_' to define a word \(hard-coded in Tk's
tkTextIndex.c\) \(in Tk8.5a0 this has been fixed to use _Tcl\_UniCharIsWordChar_\)
3. uses Tcl's regexp engine's definition of a word \(this ought to be the same as that used in \(2\)\).
4. Anything separated by white-space from something else, used with '-wrap word' to define line-wrapping in text widgets \(and canvases\).
It is quite likely that most of the above are different under some
circumstances or some platforms/locales, and certainly if the
user/developer wants to create a text widget with a different word
definition, they basically can't in any consistent way.
# Implementation
None yet.
# Copyright
This document is placed in the public domain.

TIP: 123Title: Adding an Exponentiation Operator to the [expr] Command
Version: $Revision: 1.6 $
Author: Arjen Markus <arjen.markus@wldelft.nl>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 16-Dec-2002
Post-History:
Keywords: mathematics,evaluation
Tcl-Version: 8.5
~ Abstract
This TIP proposes to add a new operator to the operators recognised by
the [[expr]] command: the exponentiation operator. This operator will
enhance the functionality of the current ''pow()'' function by
returning a result that depends on the type of its operands. It will
also make complicated formulae more readable.
~ Introduction
Currently Tcl's [[expr]] command uses the exponentiation function
''pow()'' to calculate such expressions as "2 to the power 10". The
drawback of this is twofold:
* Expressions using several exponentiations become difficult to read.
For instance, a third-degree polynomial looks like:
| 2.0*pow($x,3) - 1.2*pow($x,2) + 3.0*$x + 4.0
> or:
| 2.0*$x*$x*$x - 1.2*$x*$x + 3.0*$x + 4.0
* The result of raising an integer to an integer power is a
double: 2 to the power 10 is 1024.0, not 1024.
Other languages, like for instance FORTRAN, use an operator instead of
a function. Two operators are commonly found: ** and ^. As the
latter already has a meaning within the [[expr]] command, we propose
to add the "**" operator instead. The above example would become:
| 2.0*$x**3 - 1.2*$x**2 + 3.0*$x + 4.0
~ Mathematical Details
The implementation of the exponentiation operator will have the
following properties (below we refer to the expression ''$x**$y''):
If ''x'' and ''y'' are both integers (ordinary or wide):
* The result is of the same type as the widest operand
* An error is raised if the operation makes no mathematical sense,
''0**(-1)'' for instance.
* If ''x'' has the value 0, then:
> * if ''y > 0'', the result is 0
> * if ''y < 0'', the result is an error
> * if ''y == 0'', the result is 1
* If ''x'' has the value 1, then the result is always 1
* If ''y'' has the value 0, the result is always 1
* If ''x'' has a negative value lower than -1 and ''y < 0'', the
result is 0
* If ''x'' has the value -1, then depending on whether ''y'' is even
or odd, the result is 1 or -1 (respectively.)
* For all other combinations, the value is "''x'' raised to the power
''y''"
* When evaluating this, no attention is paid to overflow, even though
the result might fit into a wide integer (though of course the
result will be a wide integer if either operand was wide.) This is
in accordance with the type model used in other [[expr]] operators.
If either ''x'' or ''y'' is a double, the C function ''pow()'' is used
to compute the result.
The following expressions are parsed and evaluated in accordance with
all other operators:
| $x ** $y ** $z ==> ($x ** $y ) ** $z
| $x ** -1 ==> ($x ** (-1))
The precedence of the exponentiation operator is thus ''higher'' than
the multiplication, division and remainder operations and lower than
the unary operations, in accordance with common definitions.
~ Sample Implementation
http://sf.net/tracker/?func=detail&aid=655176&group_id=10894&atid=310894~ Copyright
This document is placed in the public domain.

# TIP 123: Adding an Exponentiation Operator to the [expr] Command
Author: Arjen Markus <arjen.markus@wldelft.nl>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 16-Dec-2002
Post-History:
Keywords: mathematics,evaluation
Tcl-Version: 8.5
-----# Abstract
This TIP proposes to add a new operator to the operators recognised by
the [expr] command: the exponentiation operator. This operator will
enhance the functionality of the current _pow\(\)_ function by
returning a result that depends on the type of its operands. It will
also make complicated formulae more readable.
# Introduction
Currently Tcl's [expr] command uses the exponentiation function
_pow\(\)_ to calculate such expressions as "2 to the power 10". The
drawback of this is twofold:
* Expressions using several exponentiations become difficult to read.
For instance, a third-degree polynomial looks like:
2.0*pow($x,3) - 1.2*pow($x,2) + 3.0*$x + 4.0
> or:
2.0*$x*$x*$x - 1.2*$x*$x + 3.0*$x + 4.0
* The result of raising an integer to an integer power is a
double: 2 to the power 10 is 1024.0, not 1024.
Other languages, like for instance FORTRAN, use an operator instead of
a function. Two operators are commonly found: \*\* and ^. As the
latter already has a meaning within the [expr] command, we propose
to add the "\*\*" operator instead. The above example would become:
2.0*$x**3 - 1.2*$x**2 + 3.0*$x + 4.0
# Mathematical Details
The implementation of the exponentiation operator will have the
following properties \(below we refer to the expression _$x\*\*$y_\):
If _x_ and _y_ are both integers \(ordinary or wide\):
* The result is of the same type as the widest operand
* An error is raised if the operation makes no mathematical sense,
_0\*\*\(-1\)_ for instance.
* If _x_ has the value 0, then:
> \* if _y > 0_, the result is 0
> \* if _y < 0_, the result is an error
> \* if _y == 0_, the result is 1
* If _x_ has the value 1, then the result is always 1
* If _y_ has the value 0, the result is always 1
* If _x_ has a negative value lower than -1 and _y < 0_, the
result is 0
* If _x_ has the value -1, then depending on whether _y_ is even
or odd, the result is 1 or -1 \(respectively.\)
* For all other combinations, the value is "_x_ raised to the power
_y_"
* When evaluating this, no attention is paid to overflow, even though
the result might fit into a wide integer \(though of course the
result will be a wide integer if either operand was wide.\) This is
in accordance with the type model used in other [expr] operators.
If either _x_ or _y_ is a double, the C function _pow\(\)_ is used
to compute the result.
The following expressions are parsed and evaluated in accordance with
all other operators:
$x ** $y ** $z ==> ($x ** $y ) ** $z
$x ** -1 ==> ($x ** (-1))
The precedence of the exponentiation operator is thus _higher_ than
the multiplication, division and remainder operations and lower than
the unary operations, in accordance with common definitions.
# Sample Implementation
<http://sf.net/tracker/?func=detail&aid=655176&group\_id=10894&atid=310894># Copyright
This document is placed in the public domain.

TIP: 124Title: High-Resolution Absolute Time Values From [clock]
Version: $Revision: 1.13 $
Author: Mark Harrison <mh@pixar.com>
Author: Kevin Kenny <kennykb@users.sourceforge.net>
State: Final
Type: Project
Vote: Done
Created: 20-Dec-2002
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes altering the '''clock clicks''' command to add
a '''-microseconds''' option, and to tie it to an absolute reference.
~ Rationale
The '''clock clicks''' command is a useful feature for analyzing and
tuning performance in Tcl programs. Since the timer is relative,
however, it is difficult to correlate times between two programs or
with timestamps stored in a log file.
~ Proposed change
This TIP proposes two changes to '''clock clicks''' to make it more
useful in situations where absolute times are needed:
1. Change '''clock clicks -milliseconds''' to return a wide integer,
representing the number of milliseconds since the Posix epoch.
1. Add a '''clock clicks -microseconds''' that likewise returns
a wide integer, representing the number of microseconds since
the Posix epoch.
~ Compatibility with existing code
Now that Tcl supports 64-bit integers transparently, compatibility
with existing code is unlikely to cause trouble. In practice, the
only uses of '''clock clicks''' that the authors have seen all simply
compute the difference between two successive values. Since the
difference will continue to fit in 32 bits if it did so before,
such code will simply continue to work.
The change from a relative standard to an absolute one is
likewise transparent. One fact that supports this claim is that
such a change was made in '''clock clicks -milliseconds''' on Windows
in support of [7], and no Windows users reported trouble as
a result.
Code that attempts to use '''clock clicks'' as an absolute standard
by correlating its values with those returned from '''clock seconds'''(as in http://wiki.tcl.tk/1035) will also continue to work without
without change, although the improvements to '''clock clicks''' will
certainly provide a better way to do it.
~ Reference Implementation
The original reference implementation at
http://sf.net/tracker/?func=detail&atid=310894&aid=656997&group_id=10894
presents an interface that has since been superseded. The
reference implementation will be updated at such time as this
TIP is approved.
~ Copyright
This document has been placed in the public domain.
~ Comments
Gerald Lester also points out (in the Tcl'ers Chat) that we ought
to make [clock seconds] return a wide value at the same time as
we implement the rest of these changes, in order to address the
Y2038 problem. Alas, the problem goes deeper; the ''gettimeofday()''
call (and Tcl's mirroring of it in the ''Tcl_GetTime'' interface) is
not Y2038-ready. The necessary upgrades to return a 64-bit value
for [clock seconds] are outside the scope of this TIP.
Several people have informally asked whether 64 bits are enough
to hold the microsecond counter. Since 2**64 microseconds is
roughly half a million years, the authors are confident that
the answer is, "yes."

# TIP 124: High-Resolution Absolute Time Values From [clock]
Author: Mark Harrison <mh@pixar.com>
Author: Kevin Kenny <kennykb@users.sourceforge.net>
State: Final
Type: Project
Vote: Done
Created: 20-Dec-2002
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes altering the **clock clicks** command to add
a **-microseconds** option, and to tie it to an absolute reference.
# Rationale
The **clock clicks** command is a useful feature for analyzing and
tuning performance in Tcl programs. Since the timer is relative,
however, it is difficult to correlate times between two programs or
with timestamps stored in a log file.
# Proposed change
This TIP proposes two changes to **clock clicks** to make it more
useful in situations where absolute times are needed:
1. Change **clock clicks -milliseconds** to return a wide integer,
representing the number of milliseconds since the Posix epoch.
1. Add a **clock clicks -microseconds** that likewise returns
a wide integer, representing the number of microseconds since
the Posix epoch.
# Compatibility with existing code
Now that Tcl supports 64-bit integers transparently, compatibility
with existing code is unlikely to cause trouble. In practice, the
only uses of **clock clicks** that the authors have seen all simply
compute the difference between two successive values. Since the
difference will continue to fit in 32 bits if it did so before,
such code will simply continue to work.
The change from a relative standard to an absolute one is
likewise transparent. One fact that supports this claim is that
such a change was made in **clock clicks -milliseconds** on Windows
in support of [[7]](7.md), and no Windows users reported trouble as
a result.
Code that attempts to use **clock clicks_ as an absolute standard
by correlating its values with those returned from **clock seconds**\(as in <http://wiki.tcl.tk/1035\)> will also continue to work without
without change, although the improvements to **clock clicks** will
certainly provide a better way to do it.
# Reference Implementation
The original reference implementation at
<http://sf.net/tracker/?func=detail&atid=310894&aid=656997&group\_id=10894>
presents an interface that has since been superseded. The
reference implementation will be updated at such time as this
TIP is approved.
# Copyright
This document has been placed in the public domain.
# Comments
Gerald Lester also points out \(in the Tcl'ers Chat\) that we ought
to make [clock seconds] return a wide value at the same time as
we implement the rest of these changes, in order to address the
Y2038 problem. Alas, the problem goes deeper; the _gettimeofday\(\)_
call \(and Tcl's mirroring of it in the _Tcl\_GetTime_ interface\) is
not Y2038-ready. The necessary upgrades to return a 64-bit value
for [clock seconds] are outside the scope of this TIP.
Several people have informally asked whether 64 bits are enough
to hold the microsecond counter. Since 2\*\*64 microseconds is
roughly half a million years, the authors are confident that
the answer is, "yes."

TIP: 125Title: Converting between Frame and Toplevel Windows
Version: $Revision: 1.10 $
Author: Brian Griffin <bgriffin@model.com>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Author: Sacha Schär <sacha.schaer@unibas.ch>
State: Final
Type: Project
Vote: Done
Created: 20-Jan-2003
Post-History:
Keywords: reparent,Tk
Tcl-Version: 8.5
~ Abstract
This TIP modifies the '''wm''' command to act as a geometry manager for
Frames, allowing them to become Toplevel windows.
~ Rationale
One of the usability features introduced recently in many windowing systems,
is the ability to "tear out" or "insert" sub-frames and panes within an
application. The most common form of this is the ability to tear off a
toolbar, making it an independent window. This feature gives the end user the
ability to configure the application in ways that best suit them and improve
their overall productivity.
................................................................................
Currently there is no mechanism with Tk to move a widget subtree to a new
toplevel parent without reconstructing the hierarchy from scratch. The basic
command will give the Tk programmer the ability to implement megawidgets that
support tear-off toolbars, notebooks, etc., since the children widgets of the
frame will follow along, unchanged.
An alternative approach has been taken from the orignal '''wm toplevel'''
proposal as described here:
In Tk, Toplevel windows are basically a special form of a Frame which are
managed by the window manager. The proposal is to add the commands '''wm
manage''' and '''wm forget''' which will take an arbitrary Frame and allow it
to be managed by the window manager, making it a Toplevel window.
There are three configure options of Toplevel that are not present on a frame,
'''-menu''', '''-screen''', and '''-use'''. The options '''-screen''' and
'''-use''' can only be specified when a Toplevel is created and cannot be
modified later. The '''-menu''' option can be modified after widget creation
and should be available on a frame so that when the frame is managed as a
Toplevel, the menu can be present, therefore, this option has been added to
the Frame widget. When a frame is not managed as a toplevel, the '''-menu'''
option is ignored.
Note: because of special conditions of Toplevel, it is necessary to restrict
them to only be managed by the window manager; they cannot be converted to
frames and therefore, '''wm forget''' is not allowed on a Toplevel widget.
~ Reference Implementation
TkToolkit patch #998125 implements the '''wm manage''' and '''wm forget'''
commands on Unix and Windows. A partial MacOSX implementation is also
provided. This implementation only works for buttons and menubuttons. For the
remaining Tk widgets to be present in a '''wm manage''''d frame, the widgets
need to recognize and implement the TK_URBAN_RENEWAL flag.
The original '''wm toplevel''' implementation can be obtained
[ftp://ftp.model.com/pub/tcl/reframe], with patch files as well as Linux and
Win32 builds of 8.3.4 and 8.4.1 available.
The ''demo.tcl'' file illustrates the basic function.
~ Known Issues
The reference implementation works on Windows and Linux. A Mac implementation
has been partially implemented. Because of the Mac architecture, it is
necessary to reconstruct widgets in order to remap their window hierarchy.
The reference implementation includes a widget flag, TK_URBAN_RENEWAL which is
used to trigger this reconstruction.
Because of assumptions and restrictions, Toplevels can only be managed by the
window manager and cannot become frames and be managed by any other geometry
manager. This restriction is enforced which means the '''-use''' and
'''-screen''' options do not present any problem or complication with this
proposal.
The toplevel '''-menu''' option has been added to frames. This allows the
setting of menus on a frame so that when the frame is managed as a toplevel,
the menus will appear, otherwise the option is ignored.
~ Alternatives
An alternative to extending the '''wm''' command is to make this feature
either a method or a configure option of the Toplevel and Frame widget.
''A.'' [[''w'' '''detach''']] to convert a Frame into a Toplevel, and [[''w''
attach''']] to convert a Toplevel into a Frame.
''B.'' [[''w'' '''configure -toplevel 1''']] to convert a Frame into a
Toplevel, and [[''w'' '''configure -toplevel 0''']] to convert a Toplevel into
a Frame.
These different approaches (wm, method, config) depend on how the feature is
viewed. One view is that the of changing the nature or class (Toplevel, Frame)
of the widget. An alternative view is that of changing "who" manages the
widget (wm, place, grid, pack).
I suppose another alternative is to follow the precedent of the geometry
managers:
* '''wm''' ''w'' ?'''-transient''' ''parent''? ?'''-overrideredirect'''''boolean''? ...
* '''wm forget''' ''w''~ Comments
''From Sacha Schär <sacha.schaer@unibas.ch>:''
The way the TIP and its alternatives are proposed, a frame can be made into a
toplevel and vice versa. However, it is not possible, to move a frame from one
toplevel to another. This is of importance even for the most common motivation
for this TIP, the possibility to implement a tear-off behaviour. If a frame is
going to be teared out of a toplevel window, it might be desirable that it
becomes part of another toplevel window, together with some extra decoration.
I see, that with this more general behaviour, the path name becomes a problem.
Therefore I propose the alternative approach, of making the '''-container'''
and '''-use''' options dynamically configurable as already discussed in the
past
''From Brian:''
The problem is that we can't reparent widgets without changing their Tk widget
pathname. Doing this introduces all kinds of sticky problems.
In practice, the decoration is in the form of Menus, Toolbars, and Footers.
All of these things can (must) be created and managed as part of the
dock/undock process. The way to do this is to create these things the first
time a window is undocked, then just "hide" them when redocking the window;
they still exist, but are just not mapped anywhere, until the next time the
window is undocked.
The bottom line is that the '''wm manage''' command does not do ''everything''
you need when performing an undock operation; it is only a small piece of the
pie, but a very critical one.
~ Copyright
This document has been placed in the public domain.

# TIP 125: Converting between Frame and Toplevel Windows
Author: Brian Griffin <bgriffin@model.com>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Author: Sacha SchÃ¤r <sacha.schaer@unibas.ch>
State: Final
Type: Project
Vote: Done
Created: 20-Jan-2003
Post-History:
Keywords: reparent,Tk
Tcl-Version: 8.5
-----# Abstract
This TIP modifies the **wm** command to act as a geometry manager for
Frames, allowing them to become Toplevel windows.
# Rationale
One of the usability features introduced recently in many windowing systems,
is the ability to "tear out" or "insert" sub-frames and panes within an
application. The most common form of this is the ability to tear off a
toolbar, making it an independent window. This feature gives the end user the
ability to configure the application in ways that best suit them and improve
their overall productivity.
................................................................................
Currently there is no mechanism with Tk to move a widget subtree to a new
toplevel parent without reconstructing the hierarchy from scratch. The basic
command will give the Tk programmer the ability to implement megawidgets that
support tear-off toolbars, notebooks, etc., since the children widgets of the
frame will follow along, unchanged.
An alternative approach has been taken from the orignal **wm toplevel**
proposal as described here:
In Tk, Toplevel windows are basically a special form of a Frame which are
managed by the window manager. The proposal is to add the commands **wm
manage** and **wm forget** which will take an arbitrary Frame and allow it
to be managed by the window manager, making it a Toplevel window.
There are three configure options of Toplevel that are not present on a frame,
**-menu**, **-screen**, and **-use**. The options **-screen** and
**-use** can only be specified when a Toplevel is created and cannot be
modified later. The **-menu** option can be modified after widget creation
and should be available on a frame so that when the frame is managed as a
Toplevel, the menu can be present, therefore, this option has been added to
the Frame widget. When a frame is not managed as a toplevel, the **-menu**
option is ignored.
Note: because of special conditions of Toplevel, it is necessary to restrict
them to only be managed by the window manager; they cannot be converted to
frames and therefore, **wm forget** is not allowed on a Toplevel widget.
# Reference Implementation
TkToolkit patch \#998125 implements the **wm manage** and **wm forget**
commands on Unix and Windows. A partial MacOSX implementation is also
provided. This implementation only works for buttons and menubuttons. For the
remaining Tk widgets to be present in a **wm manage**'d frame, the widgets
need to recognize and implement the TK\_URBAN\_RENEWAL flag.
The original **wm toplevel** implementation can be obtained
[ftp://ftp.model.com/pub/tcl/reframe], with patch files as well as Linux and
Win32 builds of 8.3.4 and 8.4.1 available.
The _demo.tcl_ file illustrates the basic function.
# Known Issues
The reference implementation works on Windows and Linux. A Mac implementation
has been partially implemented. Because of the Mac architecture, it is
necessary to reconstruct widgets in order to remap their window hierarchy.
The reference implementation includes a widget flag, TK\_URBAN\_RENEWAL which is
used to trigger this reconstruction.
Because of assumptions and restrictions, Toplevels can only be managed by the
window manager and cannot become frames and be managed by any other geometry
manager. This restriction is enforced which means the **-use** and
**-screen** options do not present any problem or complication with this
proposal.
The toplevel **-menu** option has been added to frames. This allows the
setting of menus on a frame so that when the frame is managed as a toplevel,
the menus will appear, otherwise the option is ignored.
# Alternatives
An alternative to extending the **wm** command is to make this feature
either a method or a configure option of the Toplevel and Frame widget.
_A._ [_w_ **detach**] to convert a Frame into a Toplevel, and [_w_
attach**] to convert a Toplevel into a Frame.
_B._ [_w_ **configure -toplevel 1**] to convert a Frame into a
Toplevel, and [_w_ **configure -toplevel 0**] to convert a Toplevel into
a Frame.
These different approaches \(wm, method, config\) depend on how the feature is
viewed. One view is that the of changing the nature or class \(Toplevel, Frame\)
of the widget. An alternative view is that of changing "who" manages the
widget \(wm, place, grid, pack\).
I suppose another alternative is to follow the precedent of the geometry
managers:
* **wm** _w_ ?**-transient** _parent_? ?**-overrideredirect**_boolean_? ...
* **wm forget** _w_# Comments
_From Sacha SchÃ¤r <sacha.schaer@unibas.ch>:_
The way the TIP and its alternatives are proposed, a frame can be made into a
toplevel and vice versa. However, it is not possible, to move a frame from one
toplevel to another. This is of importance even for the most common motivation
for this TIP, the possibility to implement a tear-off behaviour. If a frame is
going to be teared out of a toplevel window, it might be desirable that it
becomes part of another toplevel window, together with some extra decoration.
I see, that with this more general behaviour, the path name becomes a problem.
Therefore I propose the alternative approach, of making the **-container**
and **-use** options dynamically configurable as already discussed in the
past
_From Brian:_
The problem is that we can't reparent widgets without changing their Tk widget
pathname. Doing this introduces all kinds of sticky problems.
In practice, the decoration is in the form of Menus, Toolbars, and Footers.
All of these things can \(must\) be created and managed as part of the
dock/undock process. The way to do this is to create these things the first
time a window is undocked, then just "hide" them when redocking the window;
they still exist, but are just not mapped anywhere, until the next time the
window is undocked.
The bottom line is that the **wm manage** command does not do _everything_
you need when performing an undock operation; it is only a small piece of the
pie, but a very critical one.
# Copyright
This document has been placed in the public domain.

TIP: 126Title: Rich Strings for Representation Persistence
Created: 30-Jan-2003
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Type: Project
Tcl-Version: 9.0
Vote: Pending
State: Draft
Version: $Revision: 1.1 $
Post-History:
~Abstract
This TIP enriches the standard UTF-8 string representation of every
''Tcl_Obj'' to allow for improved persistence of non-string
representations. This, combined with rules for substring and
substitution handling, allows the lifetime of an object to correspond
far more closely with the more-broadly understood concept of "object
lifetime".
~Rationale
At the moment in Tcl, whenever someone wants to create a stateful
entity of limited lifespan (i.e. an object) and pass it through a Tcl
script, they have to do one of two things:
1. Tie the lifespan of the object to the lifespan of the ''Tcl_Obj''
value that represents it, so that when the internal representation
of the value is deleted, so to is the object.
2. Create a table (typically a hash table) of objects and have the
value passed through the Tcl script be a handle that is really an
index into this table. Deletion happens precisely when some kind
of delete "method" is called on the object.
Each of these techniques has problems. With the first, difficulties
arise because it is a fundamental assumption of Tcl that it is always
possible to recreate the internal-representation part of a value from
the string-representation part of the value. While this is a good
assumption for lists, numbers, etc. it is not true for anything where
the deletion of the internal-rep results in the deletion of the
underlying data structure. Thus, there is a tendency for the object
to get deleted far too early. (In practise, the problem occurs in
locations like scripts passed to Tk's [[bind]] command and in some
invocations of the [[eval]] command.) Nevertheless, this technique
can be used subject to some caveats, and this is done in a number of
extensions (e.g. TclBlend/Jacl, where Java objects are passed through
Tcl this way.)
However, the second technique has a different set of troubles.
Although the process of explicit deletion works around all the faults
with over-early deletion as described above, it instead has a strong
tendency to not delete objects at all; it is far too easy to have a
resource leak that is fairly difficult to track down. Most Tcl
extensions that deal with objects (and all the ones that predate
Tcl8.0, like [[incr Tcl]]) use this technique.
What we really need is a way to allow non-string object
representations to persist substantially longer, so making the first
of the two techniques outlined above much more robust. In particular,
I have identified the string concatenation, string substitution and
substring operations as being in need of attention, though obviously
the required work will extend further as well (the script parser is an
obvious target.) This is the focus of this TIP.
~Alterations to the Tcl_Obj
So, what is the best way of maintaining these valuable representations
within a supposedly pure-string Tcl_Obj value? Well, since we do not
want to alter the internal representation (after all, it is that which
we would really like to preserve) we will have to add an extra field
(or potentially more) to the Tcl_Obj structure itself. This is
technically feasible in a backward-compatible way, as allocation of
instances of this structure have always been performed by the Tcl
library, but there is a significant downside to this in that the
structure is far and away the most common structure allocated in the
entire Tcl core. Any field added will have a significant impact on
the overall memory consumption of any Tcl-enabled binary.
What information do we actually need to store for these
representations to be preserved in a string while allowing extraction
of the underlying values? The simplest possible option is for a list
of string-subrange/object-reference pairs to be associated with the
object, probably as an array of simple structures pointed to by a new
field of the Tcl_Obj (with a NULL indicating that no range of the
string has an object associated with it.) The easiest way of
representing those string ranges is as pairs of byte-indexes relative
to the start of the string, though character-indexes have much to
commend them (especially when working with strings with a UCS-16
internal representation) as do direct pointers into the string (easy
to compute, but much more problematic when another string is
appended.) The end of the list would be marked in some obvious way,
probably by using a NULL in the object-reference part.
This mechanism has the advantage that it keeps the increase in size of
the Tcl_Obj itself fairly small (i.e. an extra 4 bytes for another
pointer on 32-bit platforms) which is an advantage when you consider
the fact that it is likely that most strings in an average Tcl program
will not have objects contained within them. This will act to
minimise the overall memory overhead.
~Concatenation, Substitution and Substring Semantics
When two strings with these object annotations are concatenated, it is
clear that the resulting string should also have the annotations (and
the actual human-readable part will use the string-representation of
the objects), and this is trivially extended to arbitrary
concatenations of strings. Similarly for the taking of substrings
with the following restrictions:
1. Where the portion of substring taken corresponds exactly to the
part of a string associated with an object, the operation shall
instead return the object in question (which shall be assumed to
have a compatible string representation.)
2. Where a substring wholly contains a range associated with some
object, then the resulting substring object will also contain the
object associated with the "same" characters.
3. Where a substring only partially overlaps a range associated with
an object, that object will not be associated with the
corresponding characters in the resultant substring (unless the
object is separately part of the substring due to the other rules,
of course.) The object is associated with the string segment as a
whole, and not any one part of it.
Substitution, whether of variable contents, script execution results
or anything else, is semantically a concatenation operation where some
strings are (as it were) immediate operands and others are derived
from reading variables, executing scripts, etc.
Thus, if we start variable ''a'' containing an object ''Ob1'' and
variable ''b'' containing an object ''Ob2'', the following shall be
true:
|set var x${a}y${b}z => xOb1yOb2z
|# where characters 1-3 are assocaited with Ob1
|# and characters 5-7 are associated with Ob2
|set c [string range $var 1 3]
|# precisely equivalent to [set c $a]
|set d [string range $var 5 7]
|# precisely equivalent to [set d $b]
~Consequences
It is an obvious consequence of this that script evaluation should
take into account of these object annotations when attached to the
scripts themselves, particularly as the process of parsing can really
be regarded as being mostly the taking of (suitable) substrings. Only
slightly less obviously, it is also the responsibility of all code
that stores strings (and especially scripts) for future use to store
them as ''Tcl_Obj'' instances and not as just plain character strings,
and to perform any substitutions it needs to perform in a way that
preserves these object annotations on the parts that it is not
interested in. This in turn will probably require changes on the part
of many extensions to actually turn into reality.
On the other hand, there are quite a few objects (numbers and boolean
values are probably very good examples here) for which this
representation preservation is probably not a very good option as the
objects in question are perfectly preservable. It makes sense to add
some kind of signalling mechanism (e.g. a bit in a newly added
''flags'' field) to allow the type of a ''Tcl_Obj'' instance to
declare that it need not be preserved. As a general note, such a flag
would only be useful in "leaf" objects; structural objects (i.e. those
that are intended to contain others, like lists) would be expected to
do without this.
Strings (now a structural as opposed to a leaf type) probably need
even more special handling, but that can really be regarded as a
type-specific special case.
The flags field mentioned a few paragraphs above would probably have
other potential uses (e.g. for marking an object as being impossible
to change into any other type) though these lie outside the scope of
this TIP.
Because these changes at the C API level are far reaching and fairly
subtle in some cases, this TIP explicitly seeks to introduce this
behaviour with a major version number change. Although the alteration
at the script level should be small - existing code should continue
working without alteration - it is a huge philosophical leap as it
will no longer be the case that everything in Tcl will be a string, or
at least not a string that you (or anyone else) can type. Again, that
implies introduction at a new major version number.
~Notes
* The existing API function ''Tcl_InvalidateStringRep'' will gain
additional significance with the introduction of this TIP: its
invocation may well trigger the deletion of many objects.
* It is probably a very good idea indeed for code that creates
objects whose lifespan is meant to persist, to create those objects
with string representations composed entirely of alphanumeric
characters. An ideal choice might be to use a prefix derived from
the type/class of the object, and a suffix that is the address of
the object or the next value from some counter variable.
~Copyright
This document is placed in the public domain.

# TIP 126: Rich Strings for Representation Persistence
Created: 30-Jan-2003
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Type: Project
Tcl-Version: 9.0
Vote: Pending
State: Draft
Post-History:
-----# Abstract
This TIP enriches the standard UTF-8 string representation of every
_Tcl\_Obj_ to allow for improved persistence of non-string
representations. This, combined with rules for substring and
substitution handling, allows the lifetime of an object to correspond
far more closely with the more-broadly understood concept of "object
lifetime".
# Rationale
At the moment in Tcl, whenever someone wants to create a stateful
entity of limited lifespan \(i.e. an object\) and pass it through a Tcl
script, they have to do one of two things:
1. Tie the lifespan of the object to the lifespan of the _Tcl\_Obj_
value that represents it, so that when the internal representation
of the value is deleted, so to is the object.
2. Create a table \(typically a hash table\) of objects and have the
value passed through the Tcl script be a handle that is really an
index into this table. Deletion happens precisely when some kind
of delete "method" is called on the object.
Each of these techniques has problems. With the first, difficulties
arise because it is a fundamental assumption of Tcl that it is always
possible to recreate the internal-representation part of a value from
the string-representation part of the value. While this is a good
assumption for lists, numbers, etc. it is not true for anything where
the deletion of the internal-rep results in the deletion of the
underlying data structure. Thus, there is a tendency for the object
to get deleted far too early. \(In practise, the problem occurs in
locations like scripts passed to Tk's [bind] command and in some
invocations of the [eval] command.\) Nevertheless, this technique
can be used subject to some caveats, and this is done in a number of
extensions \(e.g. TclBlend/Jacl, where Java objects are passed through
Tcl this way.\)
However, the second technique has a different set of troubles.
Although the process of explicit deletion works around all the faults
with over-early deletion as described above, it instead has a strong
tendency to not delete objects at all; it is far too easy to have a
resource leak that is fairly difficult to track down. Most Tcl
extensions that deal with objects \(and all the ones that predate
Tcl8.0, like [incr Tcl]\) use this technique.
What we really need is a way to allow non-string object
representations to persist substantially longer, so making the first
of the two techniques outlined above much more robust. In particular,
I have identified the string concatenation, string substitution and
substring operations as being in need of attention, though obviously
the required work will extend further as well \(the script parser is an
obvious target.\) This is the focus of this TIP.
# Alterations to the Tcl\_Obj
So, what is the best way of maintaining these valuable representations
within a supposedly pure-string Tcl\_Obj value? Well, since we do not
want to alter the internal representation \(after all, it is that which
we would really like to preserve\) we will have to add an extra field
\(or potentially more\) to the Tcl\_Obj structure itself. This is
technically feasible in a backward-compatible way, as allocation of
instances of this structure have always been performed by the Tcl
library, but there is a significant downside to this in that the
structure is far and away the most common structure allocated in the
entire Tcl core. Any field added will have a significant impact on
the overall memory consumption of any Tcl-enabled binary.
What information do we actually need to store for these
representations to be preserved in a string while allowing extraction
of the underlying values? The simplest possible option is for a list
of string-subrange/object-reference pairs to be associated with the
object, probably as an array of simple structures pointed to by a new
field of the Tcl\_Obj \(with a NULL indicating that no range of the
string has an object associated with it.\) The easiest way of
representing those string ranges is as pairs of byte-indexes relative
to the start of the string, though character-indexes have much to
commend them \(especially when working with strings with a UCS-16
internal representation\) as do direct pointers into the string \(easy
to compute, but much more problematic when another string is
appended.\) The end of the list would be marked in some obvious way,
probably by using a NULL in the object-reference part.
This mechanism has the advantage that it keeps the increase in size of
the Tcl\_Obj itself fairly small \(i.e. an extra 4 bytes for another
pointer on 32-bit platforms\) which is an advantage when you consider
the fact that it is likely that most strings in an average Tcl program
will not have objects contained within them. This will act to
minimise the overall memory overhead.
# Concatenation, Substitution and Substring Semantics
When two strings with these object annotations are concatenated, it is
clear that the resulting string should also have the annotations \(and
the actual human-readable part will use the string-representation of
the objects\), and this is trivially extended to arbitrary
concatenations of strings. Similarly for the taking of substrings
with the following restrictions:
1. Where the portion of substring taken corresponds exactly to the
part of a string associated with an object, the operation shall
instead return the object in question \(which shall be assumed to
have a compatible string representation.\)
2. Where a substring wholly contains a range associated with some
object, then the resulting substring object will also contain the
object associated with the "same" characters.
3. Where a substring only partially overlaps a range associated with
an object, that object will not be associated with the
corresponding characters in the resultant substring \(unless the
object is separately part of the substring due to the other rules,
of course.\) The object is associated with the string segment as a
whole, and not any one part of it.
Substitution, whether of variable contents, script execution results
or anything else, is semantically a concatenation operation where some
strings are \(as it were\) immediate operands and others are derived
from reading variables, executing scripts, etc.
Thus, if we start variable _a_ containing an object _Ob1_ and
variable _b_ containing an object _Ob2_, the following shall be
true:
set var x${a}y${b}z => xOb1yOb2z
# where characters 1-3 are assocaited with Ob1
# and characters 5-7 are associated with Ob2
set c [string range $var 1 3]
# precisely equivalent to [set c $a]
set d [string range $var 5 7]
# precisely equivalent to [set d $b]
# Consequences
It is an obvious consequence of this that script evaluation should
take into account of these object annotations when attached to the
scripts themselves, particularly as the process of parsing can really
be regarded as being mostly the taking of \(suitable\) substrings. Only
slightly less obviously, it is also the responsibility of all code
that stores strings \(and especially scripts\) for future use to store
them as _Tcl\_Obj_ instances and not as just plain character strings,
and to perform any substitutions it needs to perform in a way that
preserves these object annotations on the parts that it is not
interested in. This in turn will probably require changes on the part
of many extensions to actually turn into reality.
On the other hand, there are quite a few objects \(numbers and boolean
values are probably very good examples here\) for which this
representation preservation is probably not a very good option as the
objects in question are perfectly preservable. It makes sense to add
some kind of signalling mechanism \(e.g. a bit in a newly added
_flags_ field\) to allow the type of a _Tcl\_Obj_ instance to
declare that it need not be preserved. As a general note, such a flag
would only be useful in "leaf" objects; structural objects \(i.e. those
that are intended to contain others, like lists\) would be expected to
do without this.
Strings \(now a structural as opposed to a leaf type\) probably need
even more special handling, but that can really be regarded as a
type-specific special case.
The flags field mentioned a few paragraphs above would probably have
other potential uses \(e.g. for marking an object as being impossible
to change into any other type\) though these lie outside the scope of
this TIP.
Because these changes at the C API level are far reaching and fairly
subtle in some cases, this TIP explicitly seeks to introduce this
behaviour with a major version number change. Although the alteration
at the script level should be small - existing code should continue
working without alteration - it is a huge philosophical leap as it
will no longer be the case that everything in Tcl will be a string, or
at least not a string that you \(or anyone else\) can type. Again, that
implies introduction at a new major version number.
# Notes
* The existing API function _Tcl\_InvalidateStringRep_ will gain
additional significance with the introduction of this TIP: its
invocation may well trigger the deletion of many objects.
* It is probably a very good idea indeed for code that creates
objects whose lifespan is meant to persist, to create those objects
with string representations composed entirely of alphanumeric
characters. An ideal choice might be to use a prefix derived from
the type/class of the object, and a suffix that is the address of
the object or the next value from some counter variable.
# Copyright
This document is placed in the public domain.

TIP: 127Title: Add an -index Option to [lsearch]
Version: $Revision: 1.13 $
Author: Michael Schlenker <schlenk@uni-oldenburg.de>
State: Final
Type: Project
Vote: Done
Created: 26-Feb-2003
Post-History:
Tcl-Version: 8.5
~ Abstract
Matching the ''lsort'' functionality a ''-index'' option should be
added to the ''lsearch'' command to make searching lists in list
easier. The ''lsort -index'' and the ''lsearch -index'' options should
accept list style indices like ''lindex'' and ''lset'' do. This TIP
proposes such added options.
~ Specification
Under this proposal the syntax of the ''lsearch'' is to be modified to
accept two extra options.
The ''-index'' option:
The ''lsort'' -index option would get enhanced with multiple index
support.
| lsearch ?-index index?
The ''-index'' option should work similar to the ''-index'' option
in the ''lsort'' command.
If this option is specified, each of the elements of list must itself
be a proper Tcl sublist. Instead of searching based on whole
sublists, ''lsearch'' will extract the index'th element from each
sublist and search based on the given element. The keyword end is
allowed for the index to search on the last sublist element, and
end-index searches on a sublist element offset from the end. For
example,
| lsearch -integer -index 1 {{First 24} {Second 18} {Third 30}} 18
returns ''1'', and
| lsearch -index end-1 {{a 1 e i} {b 2 3 f g} {c 4 5 6 d h}} d
returns ''2''.
The index given to ''lsearch -index'' and ''lsort -index'' may be
either a single scalar index as in the current ''lsort''
implementation or the argument to ''-index'' may be a list of scalar
indices (similar, but not exactly like the lset/lindex multiple
indices). The -index option causes lsearch and lsort to extract the
item for searching or sorting comparision from the list element by
using in effect ''lindex $element $index''.
For example:
| set record {
| {| {James Dean} {Musician} {some other data}
| }| {| {Elvis Presley} {Musician} {some more data}
| }| }|| lsearch -index {0 0} $record "Elvis"
returns ''1''.
| lsort -dictionary -decreasing -index {0 1} $record
returns ''{{{Elvis Presley} {Musician} {some more data}} {{James Dean}
{Musician} {some other data}}}''.
Note: The following example is invalid, -index only takes a list as
argument, unlike lset and lindex, that take either a list or multiple
arguments, that get concatenated by the command.
| lsearch -index 1 1 $record Presley
The ''-subindices'' option:
The ''-subindices'' option should be added only to the ''lsearch''
command as a convenience shortcut. It is only a valid option if the
''-index'' option is also used.
If ''-subindices'' is given, ''lsearch'' should return a list of
indices that can be used directly by ''lindex'' or ''lset'' to
manipulate the element used for searching, instead of the top level
element of the list.
If ''-subindices'' and ''-inline'' are specified at the same time, the
command returns the value instead of the index.
example:
| lsearch -subindices -index {0 0} $record Elvis
returns ''{1 0 0}''.
If no -all option is specified, this is the same as doing:
| concat [lsearch -index {0 0} $record Elvis] [list 0 0]
~ Rationale
Lists containing one or more level of sublists are a common technique to
simulated complex data structures like matrices or records, or for
results from database queries. The ''lsort'' command was enhanced
with the ''-index'' option for this case, to handle sorting on sublist
keys.
The ''lsearch'' command does not have this functionality yet, one has
to use ''foreach'' and ''lindex'' to loop over the list one by one.
Multiple indices for the ''-index'' option to both ''lsearch'' and
''lsort'' should be added, to make the option more similar to ''lset''
and ''lindex'' style indices.
The ''-subindices'' option is a convenience option to make constructs
like this example work well:
| foreach item [lsearch -all -subindices -index {0 0} $record bla] {
| lset record $item bar
| }
Without the subindices option it could be written as:
| foreach item [lsearch -all -index $idx $record bla] {
| lset record [concat $idx $item] bar
| }~ Reference Implementation
A reference implementation exists, see the files attached to the Tcl
Patch 693836 on SourceForge.
http://sf.net/tracker/?func=detail&atid=310894&aid=693836&group_id=10894
The reference implementation isn't honouring the -inline option if
given in conjuction with -subindices, at the moment.
~ Copyright
This document has been placed in the public domain.

# TIP 127: Add an -index Option to [lsearch]
Author: Michael Schlenker <schlenk@uni-oldenburg.de>
State: Final
Type: Project
Vote: Done
Created: 26-Feb-2003
Post-History:
Tcl-Version: 8.5
-----# Abstract
Matching the _lsort_ functionality a _-index_ option should be
added to the _lsearch_ command to make searching lists in list
easier. The _lsort -index_ and the _lsearch -index_ options should
accept list style indices like _lindex_ and _lset_ do. This TIP
proposes such added options.
# Specification
Under this proposal the syntax of the _lsearch_ is to be modified to
accept two extra options.
The _-index_ option:
The _lsort_ -index option would get enhanced with multiple index
support.
lsearch ?-index index?
The _-index_ option should work similar to the _-index_ option
in the _lsort_ command.
If this option is specified, each of the elements of list must itself
be a proper Tcl sublist. Instead of searching based on whole
sublists, _lsearch_ will extract the index'th element from each
sublist and search based on the given element. The keyword end is
allowed for the index to search on the last sublist element, and
end-index searches on a sublist element offset from the end. For
example,
lsearch -integer -index 1 {{First 24} {Second 18} {Third 30}} 18
returns _1_, and
lsearch -index end-1 {{a 1 e i} {b 2 3 f g} {c 4 5 6 d h}} d
returns _2_.
The index given to _lsearch -index_ and _lsort -index_ may be
either a single scalar index as in the current _lsort_
implementation or the argument to _-index_ may be a list of scalar
indices \(similar, but not exactly like the lset/lindex multiple
indices\). The -index option causes lsearch and lsort to extract the
item for searching or sorting comparision from the list element by
using in effect _lindex $element $index_.
For example:
set record {
{ {James Dean} {Musician} {some other data}
} { {Elvis Presley} {Musician} {some more data}
} } lsearch -index {0 0} $record "Elvis"
returns _1_.
lsort -dictionary -decreasing -index {0 1} $record
returns _\{\{\{Elvis Presley\} \{Musician\} \{some more data\}\} \{\{James Dean\}
\{Musician\} \{some other data\}\}\}_.
Note: The following example is invalid, -index only takes a list as
argument, unlike lset and lindex, that take either a list or multiple
arguments, that get concatenated by the command.
lsearch -index 1 1 $record Presley
The _-subindices_ option:
The _-subindices_ option should be added only to the _lsearch_
command as a convenience shortcut. It is only a valid option if the
_-index_ option is also used.
If _-subindices_ is given, _lsearch_ should return a list of
indices that can be used directly by _lindex_ or _lset_ to
manipulate the element used for searching, instead of the top level
element of the list.
If _-subindices_ and _-inline_ are specified at the same time, the
command returns the value instead of the index.
example:
lsearch -subindices -index {0 0} $record Elvis
returns _\{1 0 0\}_.
If no -all option is specified, this is the same as doing:
concat [lsearch -index {0 0} $record Elvis] [list 0 0]
# Rationale
Lists containing one or more level of sublists are a common technique to
simulated complex data structures like matrices or records, or for
results from database queries. The _lsort_ command was enhanced
with the _-index_ option for this case, to handle sorting on sublist
keys.
The _lsearch_ command does not have this functionality yet, one has
to use _foreach_ and _lindex_ to loop over the list one by one.
Multiple indices for the _-index_ option to both _lsearch_ and
_lsort_ should be added, to make the option more similar to _lset_
and _lindex_ style indices.
The _-subindices_ option is a convenience option to make constructs
like this example work well:
foreach item [lsearch -all -subindices -index {0 0} $record bla] {
lset record $item bar
}
Without the subindices option it could be written as:
foreach item [lsearch -all -index $idx $record bla] {
lset record [concat $idx $item] bar
}# Reference Implementation
A reference implementation exists, see the files attached to the Tcl
Patch 693836 on SourceForge.
<http://sf.net/tracker/?func=detail&atid=310894&aid=693836&group\_id=10894>
The reference implementation isn't honouring the -inline option if
given in conjuction with -subindices, at the moment.
# Copyright
This document has been placed in the public domain.

TIP: 128Title: Ability to Install a Custom Memory Allocator
Version: $Revision: 1.5 $
Author: Christophe Cap <udragon@users.sourceforge.net>
Author: Mike Jackson <hhyJackson1275@who-got-mail.com>
State: Rejected
Type: Project
Vote: Done
Created: 13-Mar-2003
Post-History:
Tcl-Version: 8.6
~ Abstract
This TIP alters Tcl to allow embedded uses of the Tcl library (and any
extensions) to either use the Tcl memory allocators as their main
allocator (especially in C++) or to set the memory allocator that Tcl
uses for itself through ''ckalloc()''.
~ Background
A while ago I was experiencing troubles when allocating images
([[image create photo]]) while memory was already exhausted, my app
crashed (due to known bug item #698571, which is in the HEAD by now!)
This shouldn't happen anyway since my application had it's new handler
set.
Tracing down the source of the allocators I noticed that Tcl uses
''HeapAlloc()'' (on Win32) to allocate its memory. Why not use
''malloc()''?
~ New/Malloc Handler
It would be nice to be able to catch memory allocation errors with a
custom new handler.
A solution could be to replace ''HeapAlloc()'' (on Win32) and other
platform specific memory handlers should be replaced by ''malloc()''.
This way a new handler can by set through ''set_new_handler()''.
Note that the Microsoft VC++ compiler has some ANSI incompatibility in
that it uses ''_set_new_handler()'' rather than ''set_new_handler()''.
We would naturally conceal this platform difference.
For example:
|#include <new>
||//
|// New handler for Microsoft Visual C++ compiler
|//
||#ifdef _MSC_VER
|#include <new.h>
||int __cdecl _newHandler(size_t size )
|{| // Do whatever
|| return 0;
|}||#else
||//
|// Ansi C/C++ new handler
|//
||void __cdecl _newHandler( void )
|{| // Do whatever
|}|#endif
||void sethandlers(void)
|{|// Microsoft compiler
|#ifdef _MSC_VER
|| _set_new_handler (_newHandler); // Setup new handler
| _set_new_mode( 1 ); // Re-route malloc failures to new handler !
||// Ansi compiler
||#else
|| set_new_handler (_newHandler); // ANSI new handler
||#endif
|}~ Tcl Implementation
The above suggested solution could work for some compilers, but may
not for all (some compilers might not support setting a malloc failure
callback.) Therefore a Tcl custom new handler functionality could be
implemented that handles Tcl specific memory allocation failures.
Something like: ''Tcl_SetMemHandler()''?
~ Copyright
This document has been placed in the public domain.

# TIP 128: Ability to Install a Custom Memory Allocator
Author: Christophe Cap <udragon@users.sourceforge.net>
Author: Mike Jackson <hhyJackson1275@who-got-mail.com>
State: Rejected
Type: Project
Vote: Done
Created: 13-Mar-2003
Post-History:
Tcl-Version: 8.6
-----# Abstract
This TIP alters Tcl to allow embedded uses of the Tcl library \(and any
extensions\) to either use the Tcl memory allocators as their main
allocator \(especially in C\+\+\) or to set the memory allocator that Tcl
uses for itself through _ckalloc\(\)_.
# Background
A while ago I was experiencing troubles when allocating images
\([image create photo]\) while memory was already exhausted, my app
crashed \(due to known bug item \#698571, which is in the HEAD by now!\)
This shouldn't happen anyway since my application had it's new handler
set.
Tracing down the source of the allocators I noticed that Tcl uses
_HeapAlloc\(\)_ \(on Win32\) to allocate its memory. Why not use
_malloc\(\)_?
# New/Malloc Handler
It would be nice to be able to catch memory allocation errors with a
custom new handler.
A solution could be to replace _HeapAlloc\(\)_ \(on Win32\) and other
platform specific memory handlers should be replaced by _malloc\(\)_.
This way a new handler can by set through _set\_new\_handler\(\)_.
Note that the Microsoft VC\+\+ compiler has some ANSI incompatibility in
that it uses _\_set\_new\_handler\(\)_ rather than _set\_new\_handler\(\)_.
We would naturally conceal this platform difference.
For example:
#include <new>
//
// New handler for Microsoft Visual C++ compiler
//
#ifdef _MSC_VER
#include <new.h>
int __cdecl _newHandler(size_t size )
{ // Do whatever
return 0;
}#else
//
// Ansi C/C++ new handler
//
void __cdecl _newHandler( void )
{ // Do whatever
}#endif
void sethandlers(void)
{// Microsoft compiler
#ifdef _MSC_VER
_set_new_handler (_newHandler); // Setup new handler
_set_new_mode( 1 ); // Re-route malloc failures to new handler !
// Ansi compiler
#else
set_new_handler (_newHandler); // ANSI new handler
#endif
}# Tcl Implementation
The above suggested solution could work for some compilers, but may
not for all \(some compilers might not support setting a malloc failure
callback.\) Therefore a Tcl custom new handler functionality could be
implemented that handles Tcl specific memory allocation failures.
Something like: _Tcl\_SetMemHandler\(\)_?
# Copyright
This document has been placed in the public domain.

TIP: 129Title: New Format Codes for the [binary] Command
Version: $Revision: 1.5 $
Author: Arjen Markus <arjen.markus@wldelft.nl>
Author: Torsten Reincke <reincke@typoscriptics.de>
State: Final
Type: Project
Vote: Done
Created: 14-Mar-2003
Post-History:
Keywords: IEEE,binary data,Tcl
Tcl-Version: 8.5
~ Abstract
This TIP proposes to add a set of new format codes to the '''binary'''
command to enhance its ability to deal with especially non-native
floating-point data. The assumption is that current limitations are
due to the distinction between little-endian and big-endian storage of
such data.
~ Introduction
The current '''binary''' command can manipulate little-endian and
big-endian ''integer'' data, but only ''native'' floating-point data.
This means that binary data from other computer systems that use a
different representation of floating-point data can not be directly
handled.
The lack of format codes to handle "native" integer data means that
one has to distinguish the current platform's byte ordering to be
platform-independent, whenever the '''binary''' command is used.
Most current computer systems use either little-endian or big-endian
byte order and the so-called IEEE representation for the exponent and
mantissa. So, the main variation to deal with is the endian-ness.
Some popular file formats, like ESRI's ArcView shape files, use both
types of byte order. It is difficult (though not impossible) to
handle these files with the current set of format codes.
It should be noted that there ''is'' more variety among floating-point
representation than just the byte order. This TIP will not solve this
more general problem.
~ Proposed Format Codes
Format codes should be available to catch the two main varieties of
byte ordering. There should, both for reasons of symmetry and for
practical purposes, also be a full set to deal with "native" data.
For integer types there are no codes to deal with native ordering. So:
* '''t''' (tiny) for short integers, using native ordering.
* '''n''' (normal) for ordinary integers, using native ordering.
* '''m''' (mirror of "w") for wide integers, using native ordering.
The floating-point types will be handled via:
* '''r'''/'''R''' (real) for single-precision reals.
* '''q'''/'''Q''' (mirror of "d") for double-precision reals.
where the lower-case is associated with little-endian order and the
upper-case with big-endian order.
~ Implementation Notes
The implementation for the integer types is simple:
* The new format codes are synonyms for the current ones, but
different ones for each endian-ness.
The implementation for the floating-point types is somewhat more
complicated, this involves adding byte swapping, if the ordering of
the platform does not correspond to that of the format code.
~ Copyright
This document is placed in the public domain.

# TIP 129: New Format Codes for the [binary] Command
Author: Arjen Markus <arjen.markus@wldelft.nl>
Author: Torsten Reincke <reincke@typoscriptics.de>
State: Final
Type: Project
Vote: Done
Created: 14-Mar-2003
Post-History:
Keywords: IEEE,binary data,Tcl
Tcl-Version: 8.5
-----# Abstract
This TIP proposes to add a set of new format codes to the **binary**
command to enhance its ability to deal with especially non-native
floating-point data. The assumption is that current limitations are
due to the distinction between little-endian and big-endian storage of
such data.
# Introduction
The current **binary** command can manipulate little-endian and
big-endian _integer_ data, but only _native_ floating-point data.
This means that binary data from other computer systems that use a
different representation of floating-point data can not be directly
handled.
The lack of format codes to handle "native" integer data means that
one has to distinguish the current platform's byte ordering to be
platform-independent, whenever the **binary** command is used.
Most current computer systems use either little-endian or big-endian
byte order and the so-called IEEE representation for the exponent and
mantissa. So, the main variation to deal with is the endian-ness.
Some popular file formats, like ESRI's ArcView shape files, use both
types of byte order. It is difficult \(though not impossible\) to
handle these files with the current set of format codes.
It should be noted that there _is_ more variety among floating-point
representation than just the byte order. This TIP will not solve this
more general problem.
# Proposed Format Codes
Format codes should be available to catch the two main varieties of
byte ordering. There should, both for reasons of symmetry and for
practical purposes, also be a full set to deal with "native" data.
For integer types there are no codes to deal with native ordering. So:
* **t** \(tiny\) for short integers, using native ordering.
* **n** \(normal\) for ordinary integers, using native ordering.
* **m** \(mirror of "w"\) for wide integers, using native ordering.
The floating-point types will be handled via:
* **r**/**R** \(real\) for single-precision reals.
* **q**/**Q** \(mirror of "d"\) for double-precision reals.
where the lower-case is associated with little-endian order and the
upper-case with big-endian order.
# Implementation Notes
The implementation for the integer types is simple:
* The new format codes are synonyms for the current ones, but
different ones for each endian-ness.
The implementation for the floating-point types is somewhat more
complicated, this involves adding byte swapping, if the ordering of
the platform does not correspond to that of the format code.
# Copyright
This document is placed in the public domain.

TIP: 13Title: Web Service for Drafting and Archiving TIPs
Version: $Revision: 1.26 $
Author: Don Porter <dgp@users.sourceforge.net>
Author: Donal K. Fellows <fellowsd@cs.man.ac.uk>
State: Accepted
Type: Process
Vote: Done
Created: 21-Nov-2000
Post-History:
~ Abstract
This document proposes the TCT provide a service on the World Wide
Web for drafting and archiving TIPs and for providing TIPs in a
variety of formats. A reference implementation is provided, and its
server requirements are outlined.
~ Background
It has been proposed (see [2]) that the TCT manage its projects and
procedures through a set of public documents known as Tcl Improvement
Proposals, or TIPs. A format for TIPs has been approved (see [3]),
and although final approval of [2] is still pending, several TIPs
have been submitted, discussed, and revised, and a few have been
approved, so acceptance of TIPs in some form seems likely.
A prototype system has been provided by Donal
Fellows <fellowsd@cs.man.ac.uk>
[http://www.cs.man.ac.uk/fellowsd-bin/TIP/]
that delivers TIPs to visitors in a variety of formats. However,
that system lacks archiving of each revision of each TIP, and
offers no interface (through the web or otherwise) for making
revisions to TIPs.
The TIP format was inspired by the format used by the Tcl'ers
Wiki [http://www.purl.org/thecliff/tcl/wiki/]. The true power
of the Tcl'ers Wiki, though, is not in the particular format it
uses, but in the fact that it empowers the whole Tcl community to
contribute to a common set of resources. The Tcl'ers Wiki
shows that valuable resources can arise out of the unrestricted
efforts of volunteers from the community.
~ Problems with Current TIP Infrastructure.
The Fellows web interface to the TIP Document Collection
[http://www.cs.man.ac.uk/fellowsd-bin/TIP/] offers valuable
browsing access to TIPs in a variety of formats. It accomplishes
the important goal of making TIPs public. However, it suffers from
two significant shortcomings:
* Revisions are only possible through the TIP Editor:
> Currently the only way to revise a TIP is to e-mail a new
revision to the TIP Editor and wait for it to replace the old
revision. As more TIPs are submitted, and as each TIP is more
frequently revised, this bottleneck will not be tolerable.
> Discussion about TIPs currently takes place in Usenet newsgroups
and on mailing lists, but because there is no easy access to
revising the TIPs themselves, the new information and viewpoints
arising in these discussions are not being folded back into the
TIPs. This means the TIPs are failing in their intended role to
present a full history of an issue to later readers. It also
means newcomers to a TIP cannot receive a full briefing in one
place, but must chase down discussions in mailing list and Usenet
archives. Few people do that, but instead repeat points already
made. The discussions about [6] reflect this problem.
* An archive of each revision of each TIP is not maintained:
> Although [2] refers to TIPs as being stored in a revision control
system, probably a CVS repository, the Fellows TIP collection is
not maintained in such a system. Since a TIP is an archive of a
public discussion of an issue, it is important to be able to
access the history of changes to each TIP.
> The ability to retrieve and restore earlier revisions of a TIP
will be especially important if public revision is permitted, so
that any TIP contents that are incorrectly removed, whether by
accident or malice, can be restored.
~ Proposal
An improved system for archiving and revising TIPs is proposed:
1. TIPs will be archived in a CVS repository.
1. Anyone with the power to call a TYANNOTT vote will have commit
access to this repository through either :pserver: or ssh
access. With this access, they will be able to revise any part
(header or body) of any TIP (whether in ''State: Draft'' or
not). Everyone having this access will be trusted to modify
TIPs only in conformance with the TIP format and the TCT
procedures.
1. An enhanced version of the Fellows TIP rendering engine will
display an [[Edit]] link at the bottom of each TIP eligible
for web-editing when that TIP is rendered in HTML.
1. For any TIP in state Draft, and for which a vote is still
pending, the [[Edit]] link will lead to an HTML form for
submitting a revised TIP. For other TIPs, no [[Edit]] link
will appear, and an attempt to directly access web-editing of
such a TIP will lead to a message stating that the TIP may not
be edited through the web interface.
1. The HTML editing form will display the TIP header, but will not
make it available for editing. The HTML form will require that
an e-mail address be entered, and will allow a name to be
................................................................................
1. The TIP rendering engine will receive the revisions, and
will use CVS client commands to merge the revisions with other
revisions and commit the revised TIP to the TIP CVS repository.
If a conflict occurs during a merge, the TIP body including the
conflicts will be returned to the user in another HTML form to
resolve the conflict.
> Note that the CVS commit function of the TIP rendering engine
implies that the CGI process in which the TIP rendering engine
runs must have a user ID with commit access to the TIP CVS
repository.
1. In the revised TIP checked in to CVS the submitter of the
revision will be added as an Author in the header of that TIP.
~ Reference Implementation
The modifications to the Fellows TIP rendering engine that add the
capabilities proposed above are now merged in. The TIP rendering
engine is maintained and publically available
[http://sourceforge.net/projects/tiprender/]. To enable the
web-editing features, set the Tcl variable FEATURE(EDIT) to 1 in
the file config.tcl. A working version of the proposed web service
is available at http://purl.org/tcl/tip/
For what it's worth, this TIP was created primarily within a web
browser, making revisions through the web interface provided by the
reference implementation.
One remaining shortcoming of the reference implementation is that it
provides no mechanism for uploading images to the TIP repository.
Images still need to be submitted through the TIP Editor, or
someone else with commit access to the TIP CVS repository.
~ Server Requirements
The reference implementation imposes the following requirements on a
server:
1. The server provides an HTTP server that serves the public
Internet, and supports the CGI interface.
1. CVS client software must be installed on the server.
1. The CVS repository containing TIPs must be on the server itself.
This is due to a CVS limitation that ''loginfo'' scripts run on
the machine housing the CVS repository, and the reference
implementation uses a ''loginfo'' script to keep the TIPs
presented through the web up to date with the commits to the
repository.
1. The CVS repository must offer commit access over the Internet
using either :pserver: or ssh to everyone with authority to call
a TYANNOTT vote.
1. The user under which the HTTP server runs its CGI processes must
have commit access to the TIP CVS repository. This may have
security implications.
~ Future Improvements
Once the TIPs are housed in a CVS repository, other services should be
easier to implement. Another browsing interface could be provided
using cvsweb [http://stud.fh-heilbronn.de/~zeller/cgi/cvsweb.cgi/] to
allow anyone in the community to browse TIP history. Another
''loginfo'' script could provide e-mail notices when a TIP is revised
to users who registered their interest in that TIP.
~ Acknowledgments
Thanks to Donal Fellows for the original TIP rendering engine and
his assistance merging in the changes. Thanks to Brent Welch for
providing the server and his assistance getting it configured for
use. Thanks to Mark Harrison for his assistance with managing
browser caching issues.
----
~ Comments from the TCT
> It might be a good idea to make the Abstract into a seperate <textarea> and
treat that specially; I've been applying the rule that a TIP's abstract
should be a single paragraph (it is implicit in the way I generate XML for
instance) and that would be much easier to enforce through this route. It
would also have the advantage of discouraging people from placing their
whole rationale in the abstract - which I've seen in several first drafts by
people who shall remain nameless - and prompting the creation of TIPs more in
keeping with the general concept of publishable documents.
> It would also be nice if each page had a way of viewing the TIP's revision
history (by a link to a suitably setup CVSweb URL?) The way that SourceForge
does its CVS-over-the-web is very nice indeed...
> ''Donal K. Fellows <fellowsd@cs.man.ac.uk>''~ Author replies to comments
Some comments I see above, or received in e-mail:
* ''Add a link to [3] on the edit page. Display [3] in a new window.''
> Link added. I don't believe in deciding to open new browser windows for
the user. If the user wants to open the link in a new window, she knows
how to do that.
* ''Add an interface to create a new TIP''
> For now, I'm trying to stick with the TIP procedures proposed in [2],
where only the TIP Editor gets to create a new TIP, so all TIPs
should still be originally submitted to him. If the TCT rejects
this proposal and adopts a different policy, we can revisit this
question.
* ''Browser caching makes it look like the edits were not made''
> Based on advice from Mark Harrison and others, I've added HTTP
headers that should prevent caching. Please try it again.
* ''I had the server time out on me''
> That's troubling. Can anyone seeing this problem provide any more
information. What were the circumstances, in detail?
* ''Add links to an interface showing revision history''
> Check out the [History] links at the bottom of the HTML rendered pages.
Thanks to Donal Fellows for coding that up. (OK, so we *did* reinvent
CVSWeb.)
* ''A separate <TEXTAREA> for the Abstract''
> This is now implemented. Please give it a try.
~ Copyright
This document has been placed in the public domain.

# TIP 13: Web Service for Drafting and Archiving TIPs
Author: Don Porter <dgp@users.sourceforge.net>
Author: Donal K. Fellows <fellowsd@cs.man.ac.uk>
State: Accepted
Type: Process
Vote: Done
Created: 21-Nov-2000
Post-History:
-----# Abstract
This document proposes the TCT provide a service on the World Wide
Web for drafting and archiving TIPs and for providing TIPs in a
variety of formats. A reference implementation is provided, and its
server requirements are outlined.
# Background
It has been proposed \(see [[2]](2.md)\) that the TCT manage its projects and
procedures through a set of public documents known as Tcl Improvement
Proposals, or TIPs. A format for TIPs has been approved \(see [[3]](3.md)\),
and although final approval of [[2]](2.md) is still pending, several TIPs
have been submitted, discussed, and revised, and a few have been
approved, so acceptance of TIPs in some form seems likely.
A prototype system has been provided by Donal
Fellows <fellowsd@cs.man.ac.uk>
<http://www.cs.man.ac.uk/fellowsd-bin/TIP/>
that delivers TIPs to visitors in a variety of formats. However,
that system lacks archiving of each revision of each TIP, and
offers no interface \(through the web or otherwise\) for making
revisions to TIPs.
The TIP format was inspired by the format used by the Tcl'ers
Wiki <http://www.purl.org/thecliff/tcl/wiki/> . The true power
of the Tcl'ers Wiki, though, is not in the particular format it
uses, but in the fact that it empowers the whole Tcl community to
contribute to a common set of resources. The Tcl'ers Wiki
shows that valuable resources can arise out of the unrestricted
efforts of volunteers from the community.
# Problems with Current TIP Infrastructure.
The Fellows web interface to the TIP Document Collection
<http://www.cs.man.ac.uk/fellowsd-bin/TIP/> offers valuable
browsing access to TIPs in a variety of formats. It accomplishes
the important goal of making TIPs public. However, it suffers from
two significant shortcomings:
* Revisions are only possible through the TIP Editor:
> Currently the only way to revise a TIP is to e-mail a new
revision to the TIP Editor and wait for it to replace the old
revision. As more TIPs are submitted, and as each TIP is more
frequently revised, this bottleneck will not be tolerable.
> Discussion about TIPs currently takes place in Usenet newsgroups
and on mailing lists, but because there is no easy access to
revising the TIPs themselves, the new information and viewpoints
arising in these discussions are not being folded back into the
TIPs. This means the TIPs are failing in their intended role to
present a full history of an issue to later readers. It also
means newcomers to a TIP cannot receive a full briefing in one
place, but must chase down discussions in mailing list and Usenet
archives. Few people do that, but instead repeat points already
made. The discussions about [[6]](6.md) reflect this problem.
* An archive of each revision of each TIP is not maintained:
> Although [[2]](2.md) refers to TIPs as being stored in a revision control
system, probably a CVS repository, the Fellows TIP collection is
not maintained in such a system. Since a TIP is an archive of a
public discussion of an issue, it is important to be able to
access the history of changes to each TIP.
> The ability to retrieve and restore earlier revisions of a TIP
will be especially important if public revision is permitted, so
that any TIP contents that are incorrectly removed, whether by
accident or malice, can be restored.
# Proposal
An improved system for archiving and revising TIPs is proposed:
1. TIPs will be archived in a CVS repository.
1. Anyone with the power to call a TYANNOTT vote will have commit
access to this repository through either :pserver: or ssh
access. With this access, they will be able to revise any part
\(header or body\) of any TIP \(whether in _State: Draft_ or
not\). Everyone having this access will be trusted to modify
TIPs only in conformance with the TIP format and the TCT
procedures.
1. An enhanced version of the Fellows TIP rendering engine will
display an [Edit] link at the bottom of each TIP eligible
for web-editing when that TIP is rendered in HTML.
1. For any TIP in state Draft, and for which a vote is still
pending, the [Edit] link will lead to an HTML form for
submitting a revised TIP. For other TIPs, no [Edit] link
will appear, and an attempt to directly access web-editing of
such a TIP will lead to a message stating that the TIP may not
be edited through the web interface.
1. The HTML editing form will display the TIP header, but will not
make it available for editing. The HTML form will require that
an e-mail address be entered, and will allow a name to be
................................................................................
1. The TIP rendering engine will receive the revisions, and
will use CVS client commands to merge the revisions with other
revisions and commit the revised TIP to the TIP CVS repository.
If a conflict occurs during a merge, the TIP body including the
conflicts will be returned to the user in another HTML form to
resolve the conflict.
> Note that the CVS commit function of the TIP rendering engine
implies that the CGI process in which the TIP rendering engine
runs must have a user ID with commit access to the TIP CVS
repository.
1. In the revised TIP checked in to CVS the submitter of the
revision will be added as an Author in the header of that TIP.
# Reference Implementation
The modifications to the Fellows TIP rendering engine that add the
capabilities proposed above are now merged in. The TIP rendering
engine is maintained and publically available
<http://sourceforge.net/projects/tiprender/> . To enable the
web-editing features, set the Tcl variable FEATURE\(EDIT\) to 1 in
the file config.tcl. A working version of the proposed web service
is available at <http://purl.org/tcl/tip/>
For what it's worth, this TIP was created primarily within a web
browser, making revisions through the web interface provided by the
reference implementation.
One remaining shortcoming of the reference implementation is that it
provides no mechanism for uploading images to the TIP repository.
Images still need to be submitted through the TIP Editor, or
someone else with commit access to the TIP CVS repository.
# Server Requirements
The reference implementation imposes the following requirements on a
server:
1. The server provides an HTTP server that serves the public
Internet, and supports the CGI interface.
1. CVS client software must be installed on the server.
1. The CVS repository containing TIPs must be on the server itself.
This is due to a CVS limitation that _loginfo_ scripts run on
the machine housing the CVS repository, and the reference
implementation uses a _loginfo_ script to keep the TIPs
presented through the web up to date with the commits to the
repository.
1. The CVS repository must offer commit access over the Internet
using either :pserver: or ssh to everyone with authority to call
a TYANNOTT vote.
1. The user under which the HTTP server runs its CGI processes must
have commit access to the TIP CVS repository. This may have
security implications.
# Future Improvements
Once the TIPs are housed in a CVS repository, other services should be
easier to implement. Another browsing interface could be provided
using cvsweb <http://stud.fh-heilbronn.de/~zeller/cgi/cvsweb.cgi/> to
allow anyone in the community to browse TIP history. Another
_loginfo_ script could provide e-mail notices when a TIP is revised
to users who registered their interest in that TIP.
# Acknowledgments
Thanks to Donal Fellows for the original TIP rendering engine and
his assistance merging in the changes. Thanks to Brent Welch for
providing the server and his assistance getting it configured for
use. Thanks to Mark Harrison for his assistance with managing
browser caching issues.
----
# Comments from the TCT
> It might be a good idea to make the Abstract into a seperate <textarea> and
treat that specially; I've been applying the rule that a TIP's abstract
should be a single paragraph \(it is implicit in the way I generate XML for
instance\) and that would be much easier to enforce through this route. It
would also have the advantage of discouraging people from placing their
whole rationale in the abstract - which I've seen in several first drafts by
people who shall remain nameless - and prompting the creation of TIPs more in
keeping with the general concept of publishable documents.
> It would also be nice if each page had a way of viewing the TIP's revision
history \(by a link to a suitably setup CVSweb URL?\) The way that SourceForge
does its CVS-over-the-web is very nice indeed...
> _Donal K. Fellows <fellowsd@cs.man.ac.uk>_# Author replies to comments
Some comments I see above, or received in e-mail:
* _Add a link to [[3]](3.md) on the edit page. Display [[3]](3.md) in a new window._ > Link added. I don't believe in deciding to open new browser windows for
the user. If the user wants to open the link in a new window, she knows
how to do that.
* _Add an interface to create a new TIP_ > For now, I'm trying to stick with the TIP procedures proposed in [[2]](2.md),
where only the TIP Editor gets to create a new TIP, so all TIPs
should still be originally submitted to him. If the TCT rejects
this proposal and adopts a different policy, we can revisit this
question.
* _Browser caching makes it look like the edits were not made_ > Based on advice from Mark Harrison and others, I've added HTTP
headers that should prevent caching. Please try it again.
* _I had the server time out on me_ > That's troubling. Can anyone seeing this problem provide any more
information. What were the circumstances, in detail?
* _Add links to an interface showing revision history_ > Check out the [History] links at the bottom of the HTML rendered pages.
Thanks to Donal Fellows for coding that up. \(OK, so we \*did\* reinvent
CVSWeb.\)
* _A separate <TEXTAREA> for the Abstract_ > This is now implemented. Please give it a try.
# Copyright
This document has been placed in the public domain.

TIP: 130Title: Unique DDE server names.
Version: $Revision: 1.4 $
Author: Pat Thoyts <patthoyts@users.sourceforge.net>
State: Final
Tcl-Version: 8.5
Created: 23-Mar-2003
Type: Project
Vote: Done
Post-History:
~ Abstract
The ''dde'' package server registration code should ensure that the
server names are unique. The proposed changes will mean end-user
visible changes for some scripts.
~ Rationale
This TIP is really driven by Bug 219293
[[http://sf.net/tracker/?func=detail&aid=219293&group_id=10894&atid=110894]]
The DDE system actually identifies servers by combination of window ID
and a service and topic string identifier. There is nothing to prevent
multiple servers having the same string identifiers but it is then a
problem to determine which server you need to communicate with.
The ''tk appname'' command handles a similar situation by making the
name unique during registration by appending a numerical suffix to the
suggested name and then returning the name registered as the result of
the command.
~ Proposed Changes
This TIP proposes to use the ''tk appname'' unique name algorithm
within the DDE server-name registration code. This TIP also proposes a
''-force'' option to this command to force the suggested name to be
registered. The ''-force'' option would then provide the current
registration scheme.
~ Reference Implementation
See the patch attached to SF patch 690354
[https://sourceforge.net/tracker/?func=detail&aid=690354&group_id=10894&atid=310894]~ Consequences
Firstly the ''dde servername'' command may register a different name
than that provided to the command. Scripts will need to check the
name actually registered by either maintaining the result of this
command, or by executing the ''dde servername'' command without any
arguments. This may impact scripts that assume the supplied argument
was in fact the name registered.
Secondly, the name registration code must obtain a list of the dde
server names currently registered on the system. It can do this by
effectively calling ''dde services TclEval''. In previous versions of
the ''dde'' package there is a potential for the ''dde services''
command to hang in the presence of applications that are not
processing messages. However, this issue has been addressed by Bug
707822's resolution.
~ Copyright
This document is hereby placed in the public domain.

# TIP 130: Unique DDE server names.
Author: Pat Thoyts <patthoyts@users.sourceforge.net>
State: Final
Tcl-Version: 8.5
Created: 23-Mar-2003
Type: Project
Vote: Done
Post-History:
-----# Abstract
The _dde_ package server registration code should ensure that the
server names are unique. The proposed changes will mean end-user
visible changes for some scripts.
# Rationale
This TIP is really driven by Bug 219293
<http://sf.net/tracker/?func=detail&aid=219293&group_id=10894&atid=110894>
The DDE system actually identifies servers by combination of window ID
and a service and topic string identifier. There is nothing to prevent
multiple servers having the same string identifiers but it is then a
problem to determine which server you need to communicate with.
The _tk appname_ command handles a similar situation by making the
name unique during registration by appending a numerical suffix to the
suggested name and then returning the name registered as the result of
the command.
# Proposed Changes
This TIP proposes to use the _tk appname_ unique name algorithm
within the DDE server-name registration code. This TIP also proposes a
_-force_ option to this command to force the suggested name to be
registered. The _-force_ option would then provide the current
registration scheme.
# Reference Implementation
See the patch attached to SF patch 690354
<https://sourceforge.net/tracker/?func=detail&aid=690354&group_id=10894&atid=310894> # Consequences
Firstly the _dde servername_ command may register a different name
than that provided to the command. Scripts will need to check the
name actually registered by either maintaining the result of this
command, or by executing the _dde servername_ command without any
arguments. This may impact scripts that assume the supplied argument
was in fact the name registered.
Secondly, the name registration code must obtain a list of the dde
server names currently registered on the system. It can do this by
effectively calling _dde services TclEval_. In previous versions of
the _dde_ package there is a potential for the _dde services_
command to hang in the presence of applications that are not
processing messages. However, this issue has been addressed by Bug
707822's resolution.
# Copyright
This document is hereby placed in the public domain.

TIP: 131Title: Read My Mind and Do What I Mean
Version: $Revision: 1.2 $
Author: Joe English <jenglish@flightlab.com>
State: Draft
Type: Project
Vote: No voting
Tcl-Version: 8.5
Created: 01-Apr-2003
Post-History:
~ Abstract
A new Tcl command is proposed, ''rmmadwim''. This is an acronym for
``Read My Mind and Do What I Mean''. This command has obvious
utility.
~ Specification
The ''rmmadwim'' command shall take no arguments. When invoked, the
Tcl interpreter shall read the programmer's mind and do what he or she
intends.
''NOTE:'' It is very important that ''rmmadwim'' read the
''programmer's'' mind, not the end user's. Otherwise the consequences
could be disastrous, since end users rarely have a firm grasp of what
the original programmer was up to.
As a consequence of this command, there is also a corresponding
function for ''expr'' which applies the same principles to general
mathematical computation:
| set result [expr {rmmadwim()}]
This extra functionality is easy to enable:
| rmmadwim
~ Rationale
What Tcl needs in order to succeed in the marketplace is a feature
that no other programming language provides, a "killer app" as it
were. The Tk toolkit, Expect, cross-platform portability, scripted
documents, tkcon, and the [[incr Tcl]] "toaster" example are all well
and good, but they have clearly failed to push Tcl usage to the point
of having critical mass. The ''rmmadwim'' command would provide a
powerful enough incentive that even Perl programmers would be
compelled to switch languages.
~ Reference Implementation
A skeletal implementation is included below. Clearly some of the
details remain to be flushed out, but this is a simple matter of
programming (SMOP). It should be a fun weekend project for Richard
Suchenwirth.
File: ''tcl/generic/tclCmdMZ.c''
Function: ''Tcl_RmmAndDWIMObjCmd''| /*ARGSUSED*/
| int
| Tcl_RmmAndDWIMObjCmd(dummy, interp, objc, objv)
| ClientData dummy; /* Not used. */
| Tcl_Interp *interp; /* Current interpreter. */
| int objc; /* Number of arguments. */
| Tcl_Obj *CONST objv[]; /* Argument objects. */
| {| int status;
| Tcl_Obj *intentions;
|| if (objc != 1) {
| Tcl_WrongNumArgs(interp, 1, objv, NULL);
| return TCL_ERROR;
| }|| status = TclReadProgrammersMind(interp, &intentions);
| if (status != TCL_OK) {
| return status;
| }|| status = TclDoWhatIsMeant(interp, intentions);
| return status;
| }~ Security Implications
It was pointed out that the ability to read the programmers' mind
carries with it certain security and privacy implementations.
To address this, the following code should be executed whenever a safe
interpreter is created:
| #| # Query the programmer's mind to obtain his or her
| # P3P settings (See "Platform for Privacy Preferences",
| # <URL: http://www.w3.org/P3P >)
| #| rmmadwim
|| #| # Disable any internal commands that are in conflict
| # with those settings:
| #| rmmadwim
In addition, the Tcl release notes should give a clear indication to
programmers about the new security implications for non-Safe
interpreters.
~ Copyright
This TIP is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License; either version 2 of
the License, or (at Richard Stallman's discretion), any later version.
Just kidding. Public domain, as usual.

# TIP 131: Read My Mind and Do What I Mean
Author: Joe English <jenglish@flightlab.com>
State: Draft
Type: Project
Vote: No voting
Tcl-Version: 8.5
Created: 01-Apr-2003
Post-History:
-----# Abstract
A new Tcl command is proposed, _rmmadwim_. This is an acronym for
\`\`Read My Mind and Do What I Mean_. This command has obvious
utility.
# Specification
The _rmmadwim_ command shall take no arguments. When invoked, the
Tcl interpreter shall read the programmer's mind and do what he or she
intends.
_NOTE:_ It is very important that _rmmadwim_ read the
_programmer's_ mind, not the end user's. Otherwise the consequences
could be disastrous, since end users rarely have a firm grasp of what
the original programmer was up to.
As a consequence of this command, there is also a corresponding
function for _expr_ which applies the same principles to general
mathematical computation:
set result [expr {rmmadwim()}]
This extra functionality is easy to enable:
rmmadwim
# Rationale
What Tcl needs in order to succeed in the marketplace is a feature
that no other programming language provides, a "killer app" as it
were. The Tk toolkit, Expect, cross-platform portability, scripted
documents, tkcon, and the [incr Tcl] "toaster" example are all well
and good, but they have clearly failed to push Tcl usage to the point
of having critical mass. The _rmmadwim_ command would provide a
powerful enough incentive that even Perl programmers would be
compelled to switch languages.
# Reference Implementation
A skeletal implementation is included below. Clearly some of the
details remain to be flushed out, but this is a simple matter of
programming \(SMOP\). It should be a fun weekend project for Richard
Suchenwirth.
File: _tcl/generic/tclCmdMZ.c_
Function: _Tcl\_RmmAndDWIMObjCmd_ /*ARGSUSED*/
int
Tcl_RmmAndDWIMObjCmd(dummy, interp, objc, objv)
ClientData dummy; /* Not used. */
Tcl_Interp *interp; /* Current interpreter. */
int objc; /* Number of arguments. */
Tcl_Obj *CONST objv[]; /* Argument objects. */
{ int status;
Tcl_Obj *intentions;
if (objc != 1) {
Tcl_WrongNumArgs(interp, 1, objv, NULL);
return TCL_ERROR;
} status = TclReadProgrammersMind(interp, &intentions);
if (status != TCL_OK) {
return status;
} status = TclDoWhatIsMeant(interp, intentions);
return status;
}# Security Implications
It was pointed out that the ability to read the programmers' mind
carries with it certain security and privacy implementations.
To address this, the following code should be executed whenever a safe
interpreter is created:
# # Query the programmer's mind to obtain his or her
# P3P settings (See "Platform for Privacy Preferences",
# <URL: http://www.w3.org/P3P >)
# rmmadwim
# # Disable any internal commands that are in conflict
# with those settings:
# rmmadwim
In addition, the Tcl release notes should give a clear indication to
programmers about the new security implications for non-Safe
interpreters.
# Copyright
This TIP is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License; either version 2 of
the License, or \(at Richard Stallman's discretion\), any later version.
Just kidding. Public domain, as usual.

TIP: 132Title: Revised Floating-Point Conversions in Tcl
Version: $Revision: 1.14 $
Author: Kevin Kenny <kennykb@acm.org>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 31-Mar-2003
Post-History:
Keywords: floating point,IEEE,precision
Tcl-Version: 8.5
~ Abstract
This TIP proposes several changes to the conversion between floating
point numbers and character strings. The changes are made to restore
the "everything is a string" contract that Tcl implicitly makes;
without them, there are observable differences in the behavior of
floating point numbers, depending on the state of the internal
representation.
~ Rationale
In today's (8.4) Tcl, there are several gaffes that make
floating-point arithmetic less useful than it might be, and cause
confusion for the users. Chief among these is that string equality
does ''not'' imply value equality for floating point numbers:
| % set tcl_precision 12
| 12
| % set a [expr 1.00000000000123]
| 1.0
| % set b [expr 1.0]
| 1.0
| % expr { $a == $b }
| 0| % expr { $a eq $b }
| 1
This behavior flies in the face of Tcl's "everything is a string"
mantra. In the interaction above, it is visible that ''a'' and ''b''
are ''not'' entirely strings; they have identical string
representations, but still test to be unequal with "==". The
underlying cause of the behavior is the fact that the default setting
for ''tcl_precision'' loses precision when converting the
floating-point number to a string. Behaviors like this have caused
Tcl's ''cognoscenti'' to recommend that all programs set
''tcl_precision'' to 17; once this setting is made, double-to-string
conversions are invertible, and everything is once again a string.
(Why 17? With IEEE-754 arithmetic, 17 decimal digits suffice to
distinguish between any two floating point numbers, no matter how
small their difference. No smaller number of digits suffices.)
Why is ''tcl_precision'' not 17 by default? The reason appears to be
that when the precision is set that high, the default IEEE-754
semantics for floating point conversions cause a certain degree of
trouble. They require that the decimal representation be the nearest
decimal representation to the value of the floating-point number that
has the given number of significant digits. This conversion gives
rise to peculiarities:
| % set tcl_precision 17; expr 0.2
| 0.20000000000000001
The peculiar behavior is, for the most part, suppressed by a lower
value for ''tcl_precision'':
| % set tcl_precision 16; expr 0.2
| 0.2
The lower value, nevertheless, introduces the trouble above. This TIP
proposes a solution to both problems.
~ Specification
This TIP proposes the following changes to the floating point
conversions in the Tcl library:
1. The default value of ''::tcl_precision'' shall be changed to 0.
A value of 0 (currently an error) shall be interpreted to mean
that a number shall be formatted using
the smallest number of decimal digits required to distinguish
it from the next floating point number above it and the next
floating point number below. Other values of ''tcl_precision''
shall continue to work as they do in Tcl 8.4.
The documentation shall formally deprecate changing
''::tcl_precision'' to any other value, warning that doing so
risks both loss of precision and inconsistency between string
equality and "==".
1. The default input conversion of floating-point numbers,
''SetDoubleFromAny'' shall be documented to guarantee precise
rounding (generally to within one-half a unit of the least
significant place [[1/2 ULP]]). IEEE-754 rounding semantics are
correct for this input. The ''strtod'' procedure from the
standard C library shall ''not'' be used for this conversion,
since so many implementations are buggy; instead, a Tcl
implementation shall be developed from scratch based on
the algorithms developed by Burger and Dybvig
[http://citeseer.ist.psu.edu/28233.html].
(It is worthy of note that several
platforms already eschew the native ''strtod'' in favour of
one provided in the ''compat/'' library, because of known bugs.)
1. When ''tcl_precision'' is zero, the output conversion
of floating-point numbers, ''UpdateStringOfDouble,'' shall convert a
floating-point number to the unique string that satisfies the
following constraints:
> * if reconverted to a binary floating point number, it yields a
number that is the closest among all strings having the given
number of significant digits.
> * if there is more than one string that is equally close but
neither string yields the given number exactly, then the string
with the even digit in the least significant place is
chosen.
> * if there is more than one string with at most the given number of
significant digits that yields the given floating-point number,
but one has fewer significant digits than the other, then the
shorter one is chosen. For example,
| % expr 1e23
> > returns ''1e+23'', not ''9.9999999999999992e+22'', even though
the latter is a nearer representation of the exact floating-point
value.
> * if there is more than one string with at most the given number of
significant digits that reconverts exactly to the same floating
point number, and all such strings are equally long, then the one
closest to the given floating point number is chosen.
> * if a floating point number lies exactly at the midpoint of two
strings with the same number of significant digits, the one
with an even digit in the least significant place is chosen.
1. The test suite for Tcl shall be upgraded to include suitable test
vectors for assessing correct rounding behavior. The paper by
Verdonk, Cuyt and Verschaeren in the References, and the
associated software, present a suitable data set for inclusion.
1. The input and output conversions shall allow for the IEEE special
values +Inf, -Inf, and NaN (and for denormalized numbers). The
[[expr]] command shall be changed to allow +Inf and -Inf as
the result of an expression; NaN shall still cause an error.
Tcl_GetDoubleFromObj shall treat +Inf and -Inf as it does any
ordinary floating point number, and return an error for NaN.
1. The [[binary scan]] and [[binary format]] commands shall explicitly
permit infinities and NaN as values.
''Donal K. Fellows notes that...'' the '''expr''' command will handle Inf and NaN quite well if asked nicely; it just won't return them.
| % expr {1 / Inf}
| 0.0
~ References
The basic principles of correctly-rounded floating point conversions
have been known for a good many years. Perhaps the two most seminal
papers on modern floating point conversion are:
> William D. Clinger, ''How to Read Floating Point Numbers
Accurately'', Proceedings of the ACM Conference on Programming
Language Design and Implementation, June 20-22 1990, pp. 92-101.
[http://citeseer.nj.nec.com/clinger90how.html]
> Guy L. Steele Jr. and Jon L. White, ''How to print floating-point
numbers accurately''. In Proceedings of the ACM Conference on
Programming Language Design and Implementation, June 20-22 1990,
pp. 112-126. [http://doi.acm.org/10.1145/93542.93559]
Both of these papers inspired David Gay to implement a library to do
correct rounding in floating point input and output conversions:
> David M. Gay, ''Correctly rounded binary-decimal and decimal-binary
conversions'', Numerical Analysis Manuscript 90-10, AT&T Bell
Laboratories, Murray Hill, New Jersey, November 1990.
[http://citeseer.nj.nec.com/gay90correctly.html]
The algorithms for output conversion that appear in the reference
implmentation are a hybrid of Gay's and the ones presented in:
> Robert G. Burger and R. Kent Dybvig, ''Printing Floating-Point Numbers
Quickly and Accurately'', SIGPLAN Conf. on Programming Language Design
and Implementation, 1996, pp. 108-116.
[http://citeseer.ist.psu.edu/28233.html]
A reasonably comprehensive set of test vectors detailing problem cases
for rounding conversions is documented in:
> Brigitte Verdonk, Annie Cuyt, Dennis Verschaeren, ''A precision and
range independent tool for testing floating-point arithmetic II:
Conversions'', ACM Transactions on Mathematical Software 27:2
(March 2001), pp. 119-140.
[http://citeseer.nj.nec.com/verdonk99precision.html]~ Reference Implementation
A partial reference implementation of this TIP is available in CVS on the
''kennykb-tcl-numerics'' branch. Since it requires additional third-party
code that is not yet in the core, a different module name is required
to check it out:
| export CVSROOT=:ext:yourname@cvs.sourceforge.net:/cvsroot/tcl
| cvs -q checkout -rkennykb-tip-132 tcl_numerics
The implementation is essentially complete; it includes changes to
the documentation and test suite needed for this TIP.
~ Acknowledgments
The discussion of just why 17 is magical was prompted by a suggestion
from Lars Hellstroem.
~ Copyright
Copyright (c) 2005 by Kevin B. Kenny. All rights reserved.
This document may be distributed subject to the terms and conditions
set forth in the Open Publication License, version 1.0
[http://www.opencontent.org/openpub/].

# TIP 132: Revised Floating-Point Conversions in Tcl
Author: Kevin Kenny <kennykb@acm.org>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 31-Mar-2003
Post-History:
Keywords: floating point,IEEE,precision
Tcl-Version: 8.5
-----# Abstract
This TIP proposes several changes to the conversion between floating
point numbers and character strings. The changes are made to restore
the "everything is a string" contract that Tcl implicitly makes;
without them, there are observable differences in the behavior of
floating point numbers, depending on the state of the internal
representation.
# Rationale
In today's \(8.4\) Tcl, there are several gaffes that make
floating-point arithmetic less useful than it might be, and cause
confusion for the users. Chief among these is that string equality
does _not_ imply value equality for floating point numbers:
% set tcl_precision 12
12
% set a [expr 1.00000000000123]
1.0
% set b [expr 1.0]
1.0
% expr { $a == $b }
0 % expr { $a eq $b }
1
This behavior flies in the face of Tcl's "everything is a string"
mantra. In the interaction above, it is visible that _a_ and _b_
are _not_ entirely strings; they have identical string
representations, but still test to be unequal with "==". The
underlying cause of the behavior is the fact that the default setting
for _tcl\_precision_ loses precision when converting the
floating-point number to a string. Behaviors like this have caused
Tcl's _cognoscenti_ to recommend that all programs set
_tcl\_precision_ to 17; once this setting is made, double-to-string
conversions are invertible, and everything is once again a string.
\(Why 17? With IEEE-754 arithmetic, 17 decimal digits suffice to
distinguish between any two floating point numbers, no matter how
small their difference. No smaller number of digits suffices.\)
Why is _tcl\_precision_ not 17 by default? The reason appears to be
that when the precision is set that high, the default IEEE-754
semantics for floating point conversions cause a certain degree of
trouble. They require that the decimal representation be the nearest
decimal representation to the value of the floating-point number that
has the given number of significant digits. This conversion gives
rise to peculiarities:
% set tcl_precision 17; expr 0.2
0.20000000000000001
The peculiar behavior is, for the most part, suppressed by a lower
value for _tcl\_precision_:
% set tcl_precision 16; expr 0.2
0.2
The lower value, nevertheless, introduces the trouble above. This TIP
proposes a solution to both problems.
# Specification
This TIP proposes the following changes to the floating point
conversions in the Tcl library:
1. The default value of _::tcl\_precision_ shall be changed to 0.
A value of 0 \(currently an error\) shall be interpreted to mean
that a number shall be formatted using
the smallest number of decimal digits required to distinguish
it from the next floating point number above it and the next
floating point number below. Other values of _tcl\_precision_
shall continue to work as they do in Tcl 8.4.
The documentation shall formally deprecate changing
_::tcl\_precision_ to any other value, warning that doing so
risks both loss of precision and inconsistency between string
equality and "==".
1. The default input conversion of floating-point numbers,
_SetDoubleFromAny_ shall be documented to guarantee precise
rounding \(generally to within one-half a unit of the least
significant place [1/2 ULP]\). IEEE-754 rounding semantics are
correct for this input. The _strtod_ procedure from the
standard C library shall _not_ be used for this conversion,
since so many implementations are buggy; instead, a Tcl
implementation shall be developed from scratch based on
the algorithms developed by Burger and Dybvig
<http://citeseer.ist.psu.edu/28233.html> .
\(It is worthy of note that several
platforms already eschew the native _strtod_ in favour of
one provided in the _compat/_ library, because of known bugs.\)
1. When _tcl\_precision_ is zero, the output conversion
of floating-point numbers, _UpdateStringOfDouble,_ shall convert a
floating-point number to the unique string that satisfies the
following constraints:
> \* if reconverted to a binary floating point number, it yields a
number that is the closest among all strings having the given
number of significant digits.
> \* if there is more than one string that is equally close but
neither string yields the given number exactly, then the string
with the even digit in the least significant place is
chosen.
> \* if there is more than one string with at most the given number of
significant digits that yields the given floating-point number,
but one has fewer significant digits than the other, then the
shorter one is chosen. For example,
% expr 1e23
> > returns _1e\+23_, not _9.9999999999999992e\+22_, even though
the latter is a nearer representation of the exact floating-point
value.
> \* if there is more than one string with at most the given number of
significant digits that reconverts exactly to the same floating
point number, and all such strings are equally long, then the one
closest to the given floating point number is chosen.
> \* if a floating point number lies exactly at the midpoint of two
strings with the same number of significant digits, the one
with an even digit in the least significant place is chosen.
1. The test suite for Tcl shall be upgraded to include suitable test
vectors for assessing correct rounding behavior. The paper by
Verdonk, Cuyt and Verschaeren in the References, and the
associated software, present a suitable data set for inclusion.
1. The input and output conversions shall allow for the IEEE special
values \+Inf, -Inf, and NaN \(and for denormalized numbers\). The
[expr] command shall be changed to allow \+Inf and -Inf as
the result of an expression; NaN shall still cause an error.
Tcl\_GetDoubleFromObj shall treat \+Inf and -Inf as it does any
ordinary floating point number, and return an error for NaN.
1. The [binary scan] and [binary format] commands shall explicitly
permit infinities and NaN as values.
_Donal K. Fellows notes that..._ the **expr** command will handle Inf and NaN quite well if asked nicely; it just won't return them.
% expr {1 / Inf}
0.0
# References
The basic principles of correctly-rounded floating point conversions
have been known for a good many years. Perhaps the two most seminal
papers on modern floating point conversion are:
> William D. Clinger, _How to Read Floating Point Numbers
Accurately_, Proceedings of the ACM Conference on Programming
Language Design and Implementation, June 20-22 1990, pp. 92-101.
<http://citeseer.nj.nec.com/clinger90how.html>
> Guy L. Steele Jr. and Jon L. White, _How to print floating-point
numbers accurately_. In Proceedings of the ACM Conference on
Programming Language Design and Implementation, June 20-22 1990,
pp. 112-126. <http://doi.acm.org/10.1145/93542.93559>
Both of these papers inspired David Gay to implement a library to do
correct rounding in floating point input and output conversions:
> David M. Gay, _Correctly rounded binary-decimal and decimal-binary
conversions_, Numerical Analysis Manuscript 90-10, AT&T Bell
Laboratories, Murray Hill, New Jersey, November 1990.
<http://citeseer.nj.nec.com/gay90correctly.html>
The algorithms for output conversion that appear in the reference
implmentation are a hybrid of Gay's and the ones presented in:
> Robert G. Burger and R. Kent Dybvig, _Printing Floating-Point Numbers
Quickly and Accurately_, SIGPLAN Conf. on Programming Language Design
and Implementation, 1996, pp. 108-116.
<http://citeseer.ist.psu.edu/28233.html>
A reasonably comprehensive set of test vectors detailing problem cases
for rounding conversions is documented in:
> Brigitte Verdonk, Annie Cuyt, Dennis Verschaeren, _A precision and
range independent tool for testing floating-point arithmetic II:
Conversions_, ACM Transactions on Mathematical Software 27:2
\(March 2001\), pp. 119-140.
<http://citeseer.nj.nec.com/verdonk99precision.html> # Reference Implementation
A partial reference implementation of this TIP is available in CVS on the
_kennykb-tcl-numerics_ branch. Since it requires additional third-party
code that is not yet in the core, a different module name is required
to check it out:
export CVSROOT=:ext:yourname@cvs.sourceforge.net:/cvsroot/tcl
cvs -q checkout -rkennykb-tip-132 tcl_numerics
The implementation is essentially complete; it includes changes to
the documentation and test suite needed for this TIP.
# Acknowledgments
The discussion of just why 17 is magical was prompted by a suggestion
from Lars Hellstroem.
# Copyright
Copyright \(c\) 2005 by Kevin B. Kenny. All rights reserved.
This document may be distributed subject to the terms and conditions
set forth in the Open Publication License, version 1.0
<http://www.opencontent.org/openpub/> .

TIP: 133Title: Extending [expr] Operators
Version: $Revision: 1.6 $
Author: Richard Suchenwirth <Richard.Suchenwirth-Bauersachs@siemens.com>
State: Draft
Type: Project
Vote: Pending
Created: 08-Apr-2003
Post-History:
Tcl-Version: 8.7
~ Abstract
This TIP proposes a way to define new operators for conditions and the
'''expr''' command. It also includes demonstrations of how it might work
in the examples: '''in''' tests inclusion in a list, and '''and''','''or''', and '''not''' are aliases for "'''&&'''", "'''||'''", "'''!'''".~ Rationale
Inclusion of a value in a list is frequently tested with the construct
| if {[lsearch -exact $list $value] >= 0} {...}
The proposal, first brought by Reinhard Max in the Tcl Chatroom, is to
allow an ''in'' operator in the language understood by '''expr''', and
the condition parts of '''for''', '''if''' and '''while''', so that
the above can be written as
| if {$value in $list} {...}
This is shorter to type and much better to read.
In the same vein, I propose to allow operators "'''and'''",
"'''or'''", "'''not'''" to be resolved exactly like the current
"'''&&'''", "'''||'''" resp "'''!'''" The new "operator aliases" are
not shorter than the original operators, but definitely better
readable - and easier typed too, as no Shift (or Alt-Gr on German
keyboards) key is needed.
When Tcl was young, almost all users knew C, so the C style operators
were a good choice. In recent years, there is tendency that Tcl is
used by persons who have no or less experience with C, but come from
other languages (etc. BASIC variants have the AND, OR, NOT operators)
or have Tcl as a first language. For all these, the option of
natural-language operators will make the learning just a little bit
easier.
~ Implementation Proposals
Donal K. Fellows remarked (on an earlier proposal relating to just an
'''in''' operator) in the Tcl Chatroom: "On the plus side, it
shouldn't be hard to implement (might need an extra opcode for
'''lsearch''', but that's pretty straightforward.)"
Pascal Scheffers proposed an extension mechanism for '''expr''' and
conditions, so the proposed extensions to the expression language can
be done in Tcl, with the commands:
| expr_register_operator in {val list} {expr {[lsearch -exact $list $val]>=0}}
| expr_register_operator and {p q} {expr {$p && $q}}
| expr_register_operator or {p q} {expr {$p || $q}}
| expr_register_operator not {p} {expr {!$p}}
Such operator registrations can have one or two arguments (for unary
and binary operators, respective) in the second argument. The third
argument is a body that is evaluated, with local variables from the
argument list substituted, and returns the resulting value, to be
substituted for the operator and its operands.
Alternatively, one could stipulate that '''expr''' interprets its
arguments in the above sense when called like this:
| expr operator in {val list} {expr {[lsearch -exact $list $val]>=0}}
| expr operator and {p q} {expr {$p && $q}}
This would currently raise an error, hence cannot break existing code.
For a simple start, it shall be an error to define an operator both as
unary and binary.
Rules for operator precedence, and a way of specifying the precedence
level, will still be needed.
A feature sometimes discussed in news:comp.lang.tcl, an assignment
operator, could in the same way easily be added by users who so want:
| expr operator = {varName value} {upvar 1 $varName var; set var $value}
Reinhard Max has also proposed a unary '''empty''' operator:
| expr operator empty {list} {expr {[llength $list]==0}}
~ Copyright
This document has been placed in the public domain.

# TIP 133: Extending [expr] Operators
Author: Richard Suchenwirth <Richard.Suchenwirth-Bauersachs@siemens.com>
State: Draft
Type: Project
Vote: Pending
Created: 08-Apr-2003
Post-History:
Tcl-Version: 8.7
-----# Abstract
This TIP proposes a way to define new operators for conditions and the
**expr** command. It also includes demonstrations of how it might work
in the examples: **in** tests inclusion in a list, and **and**,**or**, and **not** are aliases for "**&&**", "**\|\|**", "**!**".# Rationale
Inclusion of a value in a list is frequently tested with the construct
if {[lsearch -exact $list $value] >= 0} {...}
The proposal, first brought by Reinhard Max in the Tcl Chatroom, is to
allow an _in_ operator in the language understood by **expr**, and
the condition parts of **for**, **if** and **while**, so that
the above can be written as
if {$value in $list} {...}
This is shorter to type and much better to read.
In the same vein, I propose to allow operators "**and**",
"**or**", "**not**" to be resolved exactly like the current
"**&&**", "**\|\|**" resp "**!**" The new "operator aliases" are
not shorter than the original operators, but definitely better
readable - and easier typed too, as no Shift \(or Alt-Gr on German
keyboards\) key is needed.
When Tcl was young, almost all users knew C, so the C style operators
were a good choice. In recent years, there is tendency that Tcl is
used by persons who have no or less experience with C, but come from
other languages \(etc. BASIC variants have the AND, OR, NOT operators\)
or have Tcl as a first language. For all these, the option of
natural-language operators will make the learning just a little bit
easier.
# Implementation Proposals
Donal K. Fellows remarked \(on an earlier proposal relating to just an
**in** operator\) in the Tcl Chatroom: "On the plus side, it
shouldn't be hard to implement \(might need an extra opcode for
**lsearch**, but that's pretty straightforward.\)"
Pascal Scheffers proposed an extension mechanism for **expr** and
conditions, so the proposed extensions to the expression language can
be done in Tcl, with the commands:
expr_register_operator in {val list} {expr {[lsearch -exact $list $val]>=0}}
expr_register_operator and {p q} {expr {$p && $q}}
expr_register_operator or {p q} {expr {$p || $q}}
expr_register_operator not {p} {expr {!$p}}
Such operator registrations can have one or two arguments \(for unary
and binary operators, respective\) in the second argument. The third
argument is a body that is evaluated, with local variables from the
argument list substituted, and returns the resulting value, to be
substituted for the operator and its operands.
Alternatively, one could stipulate that **expr** interprets its
arguments in the above sense when called like this:
expr operator in {val list} {expr {[lsearch -exact $list $val]>=0}}
expr operator and {p q} {expr {$p && $q}}
This would currently raise an error, hence cannot break existing code.
For a simple start, it shall be an error to define an operator both as
unary and binary.
Rules for operator precedence, and a way of specifying the precedence
level, will still be needed.
A feature sometimes discussed in news:comp.lang.tcl, an assignment
operator, could in the same way easily be added by users who so want:
expr operator = {varName value} {upvar 1 $varName var; set var $value}
Reinhard Max has also proposed a unary **empty** operator:
expr operator empty {list} {expr {[llength $list]==0}}
# Copyright
This document has been placed in the public domain.

TIP: 134Title: Subsystem Per-Thread Data Interfaces
Version: $Revision: 1.4 $
Author: Colin McCormack <colin@sharedtech.dyndns.org>
State: Withdrawn
Type: Project
Vote: Pending
Created: 12-May-2003
Post-History:
Tcl-Version: 8.5
~ Abstract
Tcl core subsystems per-thread data is good, because it modularises the
core. None of the structures or access keys are exported, which is
bad, because it makes modification to these subsystems difficult in a
CVS world.
~ Rationale
If one wishes to track the CVS while writing modifications to core
subsystems one needs to minimally impact the existing source files
while adding or changing subsystem functionality. Since most
subsystem global state is kept in Thread Specific Data, and access to
this state is necessary for subsystem modifications, providing an
interface would ease coding at the core subsystem level. Modular
encapsulation would be preserved by conditionally compiling API
elements useful only to subsystem development and not intended for
extension development.
~ Justification
The degree of modularity of the core would remain unchanged, access to
subsystem TSD would presumably only be used for subsystem development,
but would enable that development to proceed more easily - one could
extend subsystems as easily as one currently writes extensions. I
think the degree of potential abuse is likely to be low.
~ Outline of Proposal
Each Thread Specific Data structure should be named uniquely for each
subsystem.
Each subsystem should have its own .h header file
Each subsystem should export a function like ''tclTSD<subsystem>''
returning pointer to its TSD. This API should be conditionally
compiled, so that it's clear that it's not available to normal
extensions.
~ Implementation
The implementation of this TIP is trivial.
~ Copyright
This document has been placed in the public domain.

# TIP 134: Subsystem Per-Thread Data Interfaces
Author: Colin McCormack <colin@sharedtech.dyndns.org>
State: Withdrawn
Type: Project
Vote: Pending
Created: 12-May-2003
Post-History:
Tcl-Version: 8.5
-----# Abstract
Tcl core subsystems per-thread data is good, because it modularises the
core. None of the structures or access keys are exported, which is
bad, because it makes modification to these subsystems difficult in a
CVS world.
# Rationale
If one wishes to track the CVS while writing modifications to core
subsystems one needs to minimally impact the existing source files
while adding or changing subsystem functionality. Since most
subsystem global state is kept in Thread Specific Data, and access to
this state is necessary for subsystem modifications, providing an
interface would ease coding at the core subsystem level. Modular
encapsulation would be preserved by conditionally compiling API
elements useful only to subsystem development and not intended for
extension development.
# Justification
The degree of modularity of the core would remain unchanged, access to
subsystem TSD would presumably only be used for subsystem development,
but would enable that development to proceed more easily - one could
extend subsystems as easily as one currently writes extensions. I
think the degree of potential abuse is likely to be low.
# Outline of Proposal
Each Thread Specific Data structure should be named uniquely for each
subsystem.
Each subsystem should have its own .h header file
Each subsystem should export a function like _tclTSD<subsystem>_
returning pointer to its TSD. This API should be conditionally
compiled, so that it's clear that it's not available to normal
extensions.
# Implementation
The implementation of this TIP is trivial.
# Copyright
This document has been placed in the public domain.

TIP: 135Title: Change 'dde servername -exact' Option to -force
Author: Pat Thoyts <patthoyts@users.sourceforge.net>
Type: Project
State: Final
Vote: Done
Version: $Revision: 1.5 $
Tcl-Version: 8.5
Keywords: dde
Created: 17-May-2003
Post-History:
~ Abstract
[130] provides for unique DDE server name registration. This TIP
renames one of the options it defines to better reflect its
behaviour.
~ Rationale
The dde extension, as modified by [130], includes an option ''-exact''
to force the name to be that specified. This TIP proposes to change
this options name to ''-force'' which better reflects its purpose. As
[130] has only just been implemented, now is the time to make this
change before it is is included in any public release of Tcl.
~ Copyright
This document is placed in the public domain.

# TIP 135: Change 'dde servername -exact' Option to -force
Author: Pat Thoyts <patthoyts@users.sourceforge.net>
Type: Project
State: Final
Vote: Done
Tcl-Version: 8.5
Keywords: dde
Created: 17-May-2003
Post-History:
-----# Abstract
[[130]](130.md) provides for unique DDE server name registration. This TIP
renames one of the options it defines to better reflect its
behaviour.
# Rationale
The dde extension, as modified by [[130]](130.md), includes an option _-exact_
to force the name to be that specified. This TIP proposes to change
this options name to _-force_ which better reflects its purpose. As
[[130]](130.md) has only just been implemented, now is the time to make this
change before it is is included in any public release of Tcl.
# Copyright
This document is placed in the public domain.

TIP: 136Title: Large List Initialisation
Version: $Revision: 1.5 $
Author: Simon Geard <simon.geard@ntlworld.com>
State: Final
Type: Project
Vote: Done
Created: 25-May-2003
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes the addition of a list initialisation command so
that large lists can be easily and efficiently initialised.
~Rationale
With the advent of the ''lset'' command in Tcl 8.4 it seems to me that
we need a method of efficiently initialising large lists that can then
be used as areas of preallocated memory. From a users point of view it
can be much easier to preallocate say a 1000 element array and then
use ''lset'' and ''lindex'' to manipulate it than using ''lappend'' to
build it up. Having posted the idea to the tcl-core mailing list
various alternatives were suggested to create a list of 1000 x
characters:
1. ''lappend'' in a loop
| unset -nocomplain s
| for {set i 0} {$i < 1000} {incr i} {lappend s {x}}
2. ''split''ting of a string
| set s [split [string repeat x 1000] ""]
3. Direct construction of the string form
| set s x[string repeat " x" 999]
None of these is particularly satisfactory. (1) seems inefficient
since there are 1000 lappend operations, (2) is not general enough
since it doesn't generalise to more than one character and (3) doesn't
actually create a list. (2) and (3) also suffer from the problem that
they are not at all obvious to new users and do nothing to dispel the
notion that "everything is a string" in Tcl.
~Implementation
I propose the introduction of a new command, ''lrepeat'':
| lrepeat <number> <element1> ?<element2>? ?<element3>? ...
which returns a list of length <number> * (number of elements). The new list is the given element sequence repeated <number> times.
<number> must be a positive integer and each <element.> a list or
string.
Examples:
| lrepeat 100 0 - returns a list of 100 zeros
| lrepeat 100 [lrepeat 100 0] - returns a 100x100 matrix (list of lists) of zeros
| lrepeat 3 a b c - returns a nine-element list {a b c a b c a b c}
| lrepeat 1 a b c - identical to [list a b c]
~Reference Implementation
I have implemented this command. A patch against the Tcl 8.4.3
source, which implements the command and provides both tests and
documentation, is available from
http://homepage.ntlworld.com/whiteowl/tcl/tcl843-lrepeat.patch
The directory also contains timing information and code which demonstrates a performance gain of over ten times for large lists.
~Copyright
This document has been placed in the public domain.

# TIP 136: Large List Initialisation
Author: Simon Geard <simon.geard@ntlworld.com>
State: Final
Type: Project
Vote: Done
Created: 25-May-2003
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes the addition of a list initialisation command so
that large lists can be easily and efficiently initialised.
# Rationale
With the advent of the _lset_ command in Tcl 8.4 it seems to me that
we need a method of efficiently initialising large lists that can then
be used as areas of preallocated memory. From a users point of view it
can be much easier to preallocate say a 1000 element array and then
use _lset_ and _lindex_ to manipulate it than using _lappend_ to
build it up. Having posted the idea to the tcl-core mailing list
various alternatives were suggested to create a list of 1000 x
characters:
1. _lappend_ in a loop
unset -nocomplain s
for {set i 0} {$i < 1000} {incr i} {lappend s {x}}
2. _split_ting of a string
set s [split [string repeat x 1000] ""]
3. Direct construction of the string form
set s x[string repeat " x" 999]
None of these is particularly satisfactory. \(1\) seems inefficient
since there are 1000 lappend operations, \(2\) is not general enough
since it doesn't generalise to more than one character and \(3\) doesn't
actually create a list. \(2\) and \(3\) also suffer from the problem that
they are not at all obvious to new users and do nothing to dispel the
notion that "everything is a string" in Tcl.
# Implementation
I propose the introduction of a new command, _lrepeat_:
lrepeat <number> <element1> ?<element2>? ?<element3>? ...
which returns a list of length <number> \* \(number of elements\). The new list is the given element sequence repeated <number> times.
<number> must be a positive integer and each <element.> a list or
string.
Examples:
lrepeat 100 0 - returns a list of 100 zeros
lrepeat 100 [lrepeat 100 0] - returns a 100x100 matrix (list of lists) of zeros
lrepeat 3 a b c - returns a nine-element list {a b c a b c a b c}
lrepeat 1 a b c - identical to [list a b c]
# Reference Implementation
I have implemented this command. A patch against the Tcl 8.4.3
source, which implements the command and provides both tests and
documentation, is available from
<http://homepage.ntlworld.com/whiteowl/tcl/tcl843-lrepeat.patch>
The directory also contains timing information and code which demonstrates a performance gain of over ten times for large lists.
# Copyright
This document has been placed in the public domain.

TIP: 137Title: Specifying Script Encodings for [source] and tclsh
Version: $Revision: 1.6 $
Author: Anton Kovalenko <a_kovalenko@fromru.com>
State: Final
Type: Project
Vote: Done
Created: 29-May-2003
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes a way to specify encoding used to read a script
with ''source'' command and (tclsh, wish) shell.~ Rationale
Now all Tcl scripts are expected to be in the system encoding. An
author of a script or a package can't specify concrete encoding to
source the script or the files of package. Common practise is to
assume that encoding to be superset of ASCII, and not to use non-ASCII
characters in the scripts that are targeting the wide audience of
potential users.
There's a way to specify UNICODE literals with \uXXXX sequences. But
it's not an universal and convenient way - this sequences aren't
substituted in {}-quoted strings, and they can't be edited in WYSIWYG
editors without special (and hence uncommon) support.
This TIP proposes to add the ''-encoding'' option support to
''source'' command and to tclsh and wish. Thus, package authors will be able
to specify encoding of the package files in ''pkgIndex.tcl''; and
script authors will be able to specify the script encoding when
calling tclsh (either in the first line beginning with ''#!'' or in
the line where the ''exec tclsh "$0" "$@"'' is located).
This TIP also proposes to use utf-8 for all the standard system
scripts, for ''pkgIndex.tcl'' and ''tclIndex'' files. Now they all are
supposed to be in the system encoding (message catalogs do not need
this support, as they are already always loaded using the UTF-8
encoding scheme), and it could event prevent Tcl itself from running
when system encoding is not a compatible superset of ASCII.
~ Specification
Tclsh will allow the encoding to be specified on the command-line in
two forms: the first form is ''-encoding name'' as two separate
arguments, and the second is ''-e:name'' (a single argument.)
The second form is intended for when the script begins with ''#!'' and
is because Unix kernels pass extra parameters from the ''#!'' line as
a single argument. This very short notation (-e:) is chosen because
some Unices limit the ''#!'' line to the length of 32.
To implement all these options, this TIP proposes a new C-level public API function Tcl_FSEvalFileEx, which is similar to Tcl_FSEvalFile, but
takes one extra argument that must be an encoding name or NULL (to use
system encoding).
| int Tcl_FSEvalFileEx(Tcl_Interp *interp, Tcl_Obj *pathPtr,
| CONST char *encodingName);
Common use of this new options will be like this:
1. In a script:
| !#/usr/bin/tclsh -e:utf-8
| do something...
> or it could be:
| #!/bin/sh
| # I wish to use tclsh \
| exec tclsh -encoding utf-8 "$0" "$@"
2. In a ''pkgIndex.tcl'':
| package ifneeded pkg 1.02 [list source -encoding utf-8 \
| [file join $dir pkg.tcl]]
~ Implementation
The partial implementation of this TIP can be found at
http://sourceforge.net/projects/tcl as Patch #742683. Some
fixes to source system scripts and pkgIndex'es in utf-8 are not
yet there. But with the existing implementation of "-encoding",
it would be really easy to implement.
~ Copyright
This document is placed in the public domain.

# TIP 137: Specifying Script Encodings for [source] and tclsh
Author: Anton Kovalenko <a_kovalenko@fromru.com>
State: Final
Type: Project
Vote: Done
Created: 29-May-2003
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes a way to specify encoding used to read a script
with _source_ command and \(tclsh, wish\) shell.# Rationale
Now all Tcl scripts are expected to be in the system encoding. An
author of a script or a package can't specify concrete encoding to
source the script or the files of package. Common practise is to
assume that encoding to be superset of ASCII, and not to use non-ASCII
characters in the scripts that are targeting the wide audience of
potential users.
There's a way to specify UNICODE literals with \\uXXXX sequences. But
it's not an universal and convenient way - this sequences aren't
substituted in \{\}-quoted strings, and they can't be edited in WYSIWYG
editors without special \(and hence uncommon\) support.
This TIP proposes to add the _-encoding_ option support to
_source_ command and to tclsh and wish. Thus, package authors will be able
to specify encoding of the package files in _pkgIndex.tcl_; and
script authors will be able to specify the script encoding when
calling tclsh \(either in the first line beginning with _\#!_ or in
the line where the _exec tclsh "$0" "$@"_ is located\).
This TIP also proposes to use utf-8 for all the standard system
scripts, for _pkgIndex.tcl_ and _tclIndex_ files. Now they all are
supposed to be in the system encoding \(message catalogs do not need
this support, as they are already always loaded using the UTF-8
encoding scheme\), and it could event prevent Tcl itself from running
when system encoding is not a compatible superset of ASCII.
# Specification
Tclsh will allow the encoding to be specified on the command-line in
two forms: the first form is _-encoding name_ as two separate
arguments, and the second is _-e:name_ \(a single argument.\)
The second form is intended for when the script begins with _\#!_ and
is because Unix kernels pass extra parameters from the _\#!_ line as
a single argument. This very short notation \(-e:\) is chosen because
some Unices limit the _\#!_ line to the length of 32.
To implement all these options, this TIP proposes a new C-level public API function Tcl\_FSEvalFileEx, which is similar to Tcl\_FSEvalFile, but
takes one extra argument that must be an encoding name or NULL \(to use
system encoding\).
int Tcl_FSEvalFileEx(Tcl_Interp *interp, Tcl_Obj *pathPtr,
CONST char *encodingName);
Common use of this new options will be like this:
1. In a script:
!#/usr/bin/tclsh -e:utf-8
do something...
> or it could be:
#!/bin/sh
# I wish to use tclsh \
exec tclsh -encoding utf-8 "$0" "$@"
2. In a _pkgIndex.tcl_:
package ifneeded pkg 1.02 [list source -encoding utf-8 \
[file join $dir pkg.tcl]]
# Implementation
The partial implementation of this TIP can be found at
<http://sourceforge.net/projects/tcl> as Patch \#742683. Some
fixes to source system scripts and pkgIndex'es in utf-8 are not
yet there. But with the existing implementation of "-encoding",
it would be really easy to implement.
# Copyright
This document is placed in the public domain.

TIP: 138Title: New TCL_HASH_KEY_SYSTEM_HASH option for Tcl hash tables
Version: $Revision: 1.5 $
Author: Kevin Kenny <kennykb@acm.org>
Author: Joe Mistachkin <joe@mistachkin.com>
State: Final
Type: Project
Vote: Done
Created: 29-May-2003
Post-History:
Keywords: thread specific data, hash table, memory allocation
Tcl-Version: 8.5
~ Abstract
This TIP proposes a new flag in the Tcl hash table API in support of a
proposed rework of thread-specific data management.
~ Introduction
The Tcl hash table constructor, ''Tcl_InitCustomHashTable'', provides
a reasonably flexible means of creating a hash table that is managed
by the application. It allows for custom procedures to calculate hash
values for keys, to compare keys for equality, and to allocate and
free hash table entries.
It always, however, allocates the hash table itself by means of
''ckalloc'' and ''ckfree''. This use of Tcl's main allocator is
normally not a problem, but a recent application of custom hash tables
has exposed the need, from time to time, of using the system allocator
(''TclpSysAlloc'' and ''TclpSysFree'') directly.
The motivating problem is that thread-specific data on certain
versions of Windows is limited to only a small number of blocks per
process (64 on at least one version). The proposed fix, which Joe
Mistachkin is pursuing, involves replacing the system calls for
managing thread-local storage with calls that manage the
thread-specific data blocks in a per-thread hash table.
Reviewing and testing the change revealed an unfortunate circular
dependency. A call to ''Tcl_InitCustomHashTable'' would attempt to
allocate the array of hash buckets by a call to ''ckalloc'' - which
translates to ''TclpAlloc''. On a threaded build, the first thing
done in this routine is to retrieve the allocation cache via
''TclpGetAllocCache.'' This in turn, once we avoid the use of
''TlsAlloc'', winds up trying to create a thread-specific data block
to hold the allocation cache. The new thread-specific data manager in
turn must allocate a hash table for the thread-specific data blocks.
The outcome is endless recursion.
The fix for the problem is simple - have either the allocation cache,
the thread-specific data hash, or both, allocated off the system heap
rather than the thread-specific allocation cache. Unfortunately,
''Tcl_InitCustomHashTable'' provides no way to accomplish this.
~ Proposal
The ''flags'' word in the ''Tcl_HashKeyType'' data structure shall be
augmented with an additional value, ''TCL_HASH_KEY_SYSTEM_HASH''. If
this bit is set in the structure passed to ''Tcl_InitCustomHashTable'',
then all memory allocated internally to manage the hash keys shall be
obtained via direct calls to ''TclpSysAlloc'', ''TclpSysRealloc'', and
''TclpSysFree'' rather than ''Tcl_Alloc'', ''Tcl_Realloc'' and
''Tcl_Free.''~ Alternatives
The authors considered and rejected the alternative of advancing
''TCL_HASH_KEY_TYPE_VERSION'' and defining in the structure three new
function pointers to allocate, reallocate, and free memory blocks.
The additional complexity (and associated performance degradation)
associated with dealing with two structure versions appeared to be
unnecessary; it is difficult to imagine a situation where any
allocator other than the system allocator or Tcl's own will be desired
for the 'buckets' array, which varies widely in size. Custom
small-block allocators make much more sense for the hash values than
they do for the table of hash buckets.
~ Implementation Notes
An implementation of this change is available from the SourceForge
patch manager as item 731356. It is part of a larger overhaul of the
thread-specific data manager.
~ Copyright
This document is placed in the public domain.

# TIP 138: New TCL_HASH_KEY_SYSTEM_HASH option for Tcl hash tables
Author: Kevin Kenny <kennykb@acm.org>
Author: Joe Mistachkin <joe@mistachkin.com>
State: Final
Type: Project
Vote: Done
Created: 29-May-2003
Post-History:
Keywords: thread specific data, hash table, memory allocation
Tcl-Version: 8.5
-----# Abstract
This TIP proposes a new flag in the Tcl hash table API in support of a
proposed rework of thread-specific data management.
# Introduction
The Tcl hash table constructor, _Tcl\_InitCustomHashTable_, provides
a reasonably flexible means of creating a hash table that is managed
by the application. It allows for custom procedures to calculate hash
values for keys, to compare keys for equality, and to allocate and
free hash table entries.
It always, however, allocates the hash table itself by means of
_ckalloc_ and _ckfree_. This use of Tcl's main allocator is
normally not a problem, but a recent application of custom hash tables
has exposed the need, from time to time, of using the system allocator
\(_TclpSysAlloc_ and _TclpSysFree_\) directly.
The motivating problem is that thread-specific data on certain
versions of Windows is limited to only a small number of blocks per
process \(64 on at least one version\). The proposed fix, which Joe
Mistachkin is pursuing, involves replacing the system calls for
managing thread-local storage with calls that manage the
thread-specific data blocks in a per-thread hash table.
Reviewing and testing the change revealed an unfortunate circular
dependency. A call to _Tcl\_InitCustomHashTable_ would attempt to
allocate the array of hash buckets by a call to _ckalloc_ - which
translates to _TclpAlloc_. On a threaded build, the first thing
done in this routine is to retrieve the allocation cache via
_TclpGetAllocCache._ This in turn, once we avoid the use of
_TlsAlloc_, winds up trying to create a thread-specific data block
to hold the allocation cache. The new thread-specific data manager in
turn must allocate a hash table for the thread-specific data blocks.
The outcome is endless recursion.
The fix for the problem is simple - have either the allocation cache,
the thread-specific data hash, or both, allocated off the system heap
rather than the thread-specific allocation cache. Unfortunately,
_Tcl\_InitCustomHashTable_ provides no way to accomplish this.
# Proposal
The _flags_ word in the _Tcl\_HashKeyType_ data structure shall be
augmented with an additional value, _TCL\_HASH\_KEY\_SYSTEM\_HASH_. If
this bit is set in the structure passed to _Tcl\_InitCustomHashTable_,
then all memory allocated internally to manage the hash keys shall be
obtained via direct calls to _TclpSysAlloc_, _TclpSysRealloc_, and
_TclpSysFree_ rather than _Tcl\_Alloc_, _Tcl\_Realloc_ and
_Tcl\_Free._# Alternatives
The authors considered and rejected the alternative of advancing
_TCL\_HASH\_KEY\_TYPE\_VERSION_ and defining in the structure three new
function pointers to allocate, reallocate, and free memory blocks.
The additional complexity \(and associated performance degradation\)
associated with dealing with two structure versions appeared to be
unnecessary; it is difficult to imagine a situation where any
allocator other than the system allocator or Tcl's own will be desired
for the 'buckets' array, which varies widely in size. Custom
small-block allocators make much more sense for the hash values than
they do for the table of hash buckets.
# Implementation Notes
An implementation of this change is available from the SourceForge
patch manager as item 731356. It is part of a larger overhaul of the
thread-specific data manager.
# Copyright
This document is placed in the public domain.

TIP: 139Title: Publish Part of Tcl's Namespace API
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Created: 03-Jun-2003
Type: Project
State: Final
Vote: Done
Tcl-Version: 8.5
Version: $Revision: 1.5 $
Post-History:
~ Abstract
This TIP makes the simpler parts of Tcl's Namespace API available to
the general C-coding public.
~ Rationale
Tcl has had namespace support for a number of years now, ever since
version 8.0. However, the C API for working with namespaces (as
opposed to the script-level [[namespace]] command) has been hidden
inside Tcl's private header files for all that time. This is mainly
because of the complexity of some of the interfaces; name resolution
is definitely difficult to document.
But about half the API is not so encumbered, being the parts that deal
with basic features like creation and destruction of namespaces
themselves. It is these simpler parts that this TIP exposes as they
have little risk of changing in the future, and I believe they should
be made public particularly as it would allow extensions that create
commands in namespaces to ensure the existence of the namespace first
without resorting to applying ''Tcl_Eval'' to a small script.
This TIP does not call for any alteration to the name, signature or
behaviour of any API function at all. Note that these functions
already have "public" names.
~ Exposed API Functions
There are the following new public functions (with signatures included
for reference only):
Tcl_CreateNamespace: Creates a new namespace.
| Tcl_Namespace * Tcl_CreateNamespace(Tcl_Interp *interp, CONST char *name,
| ClientData clientData, Tcl_NamespaceDeleteProc *deleteProc)
Tcl_DeleteNamespace: Deletes an existing namespace.
| void Tcl_DeleteNamespace(Tcl_Namespace *nsPtr)
Tcl_AppendExportList: Retrieves the export patterns for a namespace.
| int Tcl_AppendExportList(Tcl_Interp *interp, Tcl_Namespace *nsPtr,
| Tcl_Obj *objPtr)
Tcl_Export: Set/append to the export patterns for a namespace.
| int Tcl_Export(Tcl_Interp *interp, Tcl_Namespace *nsPtr,
| CONST char *pattern, int resetListFirst)
Tcl_Import: Import commands matching a pattern into a namespace.
| int Tcl_Import(Tcl_Interp *interp, Tcl_Namespace *nsPtr,
| CONST char *pattern, int allowOverwrite)
Tcl_ForgetImport: Remove imports matching a pattern.
| int Tcl_ForgetImport(Tcl_Interp *interp, Tcl_Namespace *nsPtr,
| CONST char *pattern)
Tcl_GetCurrentNamespace: Retrieve the current namespace.
| Tcl_Namespace * Tcl_GetCurrentNamespace(Tcl_Interp *interp)
Tcl_GetGlobalNamespace: Retrieve the global namespace.
| Tcl_Namespace * Tcl_GetGlobalNamespace(Tcl_Interp *interp)
Tcl_FindNamespace: Search for a namespace.
| Tcl_Namespace * Tcl_FindNamespace(Tcl_Interp *interp, CONST char *name,
| Tcl_Namespace *contextNsPtr, int flags)
Tcl_FindCommand: Search for a command and return its token,
optionally in a namespace. Note that command tokens are already
returned from ''Tcl_CreateCommand'' and ''Tcl_CreateObjCommand''
so this is not a new type of result.
| Tcl_Command Tcl_FindCommand(Tcl_Interp *interp, CONST char *name,
| Tcl_Namespace *contextNsPtr, int flags)
Tcl_GetCommandFromObj: Get a command token given a name.
| Tcl_Command Tcl_GetCommandFromObj(Tcl_Interp *interp, Tcl_Obj *objPtr)
Tcl_GetCommandFullName: Get the full name of a command from its token.
| void Tcl_GetCommandFullName(Tcl_Interp *interp, Tcl_Command command,
| Tcl_Obj *objPtr)
Each of these functions will be declared in future within
''tcl.decls'' (with the entries in ''tclInt.decls'' deprecated) and
will also be documented.
~ Copyright
This document is placed in the public domain.

# TIP 139: Publish Part of Tcl's Namespace API
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Created: 03-Jun-2003
Type: Project
State: Final
Vote: Done
Tcl-Version: 8.5
Post-History:
-----# Abstract
This TIP makes the simpler parts of Tcl's Namespace API available to
the general C-coding public.
# Rationale
Tcl has had namespace support for a number of years now, ever since
version 8.0. However, the C API for working with namespaces \(as
opposed to the script-level [namespace] command\) has been hidden
inside Tcl's private header files for all that time. This is mainly
because of the complexity of some of the interfaces; name resolution
is definitely difficult to document.
But about half the API is not so encumbered, being the parts that deal
with basic features like creation and destruction of namespaces
themselves. It is these simpler parts that this TIP exposes as they
have little risk of changing in the future, and I believe they should
be made public particularly as it would allow extensions that create
commands in namespaces to ensure the existence of the namespace first
without resorting to applying _Tcl\_Eval_ to a small script.
This TIP does not call for any alteration to the name, signature or
behaviour of any API function at all. Note that these functions
already have "public" names.
# Exposed API Functions
There are the following new public functions \(with signatures included
for reference only\):
Tcl\_CreateNamespace: Creates a new namespace.
Tcl_Namespace * Tcl_CreateNamespace(Tcl_Interp *interp, CONST char *name,
ClientData clientData, Tcl_NamespaceDeleteProc *deleteProc)
Tcl\_DeleteNamespace: Deletes an existing namespace.
void Tcl_DeleteNamespace(Tcl_Namespace *nsPtr)
Tcl\_AppendExportList: Retrieves the export patterns for a namespace.
int Tcl_AppendExportList(Tcl_Interp *interp, Tcl_Namespace *nsPtr,
Tcl_Obj *objPtr)
Tcl\_Export: Set/append to the export patterns for a namespace.
int Tcl_Export(Tcl_Interp *interp, Tcl_Namespace *nsPtr,
CONST char *pattern, int resetListFirst)
Tcl\_Import: Import commands matching a pattern into a namespace.
int Tcl_Import(Tcl_Interp *interp, Tcl_Namespace *nsPtr,
CONST char *pattern, int allowOverwrite)
Tcl\_ForgetImport: Remove imports matching a pattern.
int Tcl_ForgetImport(Tcl_Interp *interp, Tcl_Namespace *nsPtr,
CONST char *pattern)
Tcl\_GetCurrentNamespace: Retrieve the current namespace.
Tcl_Namespace * Tcl_GetCurrentNamespace(Tcl_Interp *interp)
Tcl\_GetGlobalNamespace: Retrieve the global namespace.
Tcl_Namespace * Tcl_GetGlobalNamespace(Tcl_Interp *interp)
Tcl\_FindNamespace: Search for a namespace.
Tcl_Namespace * Tcl_FindNamespace(Tcl_Interp *interp, CONST char *name,
Tcl_Namespace *contextNsPtr, int flags)
Tcl\_FindCommand: Search for a command and return its token,
optionally in a namespace. Note that command tokens are already
returned from _Tcl\_CreateCommand_ and _Tcl\_CreateObjCommand_
so this is not a new type of result.
Tcl_Command Tcl_FindCommand(Tcl_Interp *interp, CONST char *name,
Tcl_Namespace *contextNsPtr, int flags)
Tcl\_GetCommandFromObj: Get a command token given a name.
Tcl_Command Tcl_GetCommandFromObj(Tcl_Interp *interp, Tcl_Obj *objPtr)
Tcl\_GetCommandFullName: Get the full name of a command from its token.
void Tcl_GetCommandFullName(Tcl_Interp *interp, Tcl_Command command,
Tcl_Obj *objPtr)
Each of these functions will be declared in future within
_tcl.decls_ \(with the entries in _tclInt.decls_ deprecated\) and
will also be documented.
# Copyright
This document is placed in the public domain.

TIP: 14Title: Access to Tk Photo Image Transparency
Version: $Revision: 2.5 $
Author: Donal K. Fellows <fellowsd@cs.man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 22-Nov-2000
Post-History:
Keywords: Tk,photo,transparency,internal,access
Tcl-Version: 8.4.0
~ Abstract
It is useful for some code (both extensions and scripts) to have
access to the transparency information in photo images for various
reasons, but this is not currently available, even via an internal
structure defined in ''generic/tkInt.h''. This TIP is aimed at making
the information available, and in such a way at the C level that
backward compatibility is maintained in the future even if the
internal structure definitions change.
~ Rationale
I have been working for several years (on-and-off) on an extension for
Tk that allows it to have non-rectangular windows
(http://www.cs.man.ac.uk/~fellowsd/tcl/shapeidx.html) which is an
effect that is great for all sorts of purposes but which comes
particularly into its own when used in conjunction with drag-and-drop
to make drag tokens that obscure only part of what lies underneath
them. However, one of the most useful ways of specifying the shape of
a window turns out to be via images of various kinds, and the natural
way to do this is with the transparency data within the image. The
problem is that this data is locked up entirely within structures that
are completely private to ''generic/tkImgPhoto.c''; none of it is
visible at all anywhere else, even within the core. (There is code
that uses colour data instead to do this sort of trick,
http://www.sys.uea.ac.uk/~fuzz/tktrans/default.html, but this is a
slow process and frankly a little strange if we already have
transparency data available.)
There is also a more general need for scripts to be able to discover
more about the transparent areas of a photo image; at the moment,
there is no access at all to that information at the script level.
~ Changes to the C-Level API
To get around this problem, the data member ''validRegion'' of the
''PhotoMaster'' structure needs to be made available by some
mechanism. There are two ways of doing this:
1. Placing the ''PhotoMaster'' structure, or some version of it, in
''generic/tkInt.h'', or
2. Creating a function to access the data member.
The first way is very cheap initially, but also very inflexible and
creates yet another hidden version dependency (such as is tackled in
[5]) should we decide to change the structure for any reason (we also
have had problems with this sort of thing in the past in relation to
the ''Tcl_Interp'' member ''result'', direct access to which has been
deprecated for years, but where there is still existing code that does
it and which forms one of the largest barriers for some extensions
from upgrading to Tcl 8.0 or later.) It is also unnecessary since
only the core needs to know how to create new instances of the
structure.
The second way, by contrast, is far more flexible in the future as it
will allow us to completely change the internal implementation of
photo image transparency without affecting any extensions at all. The
cost of doing this is that a new entry in one of the stub tables must
be created. Due to the fact that the type of the ''validRegion''
member is (currently) internal, I propose adding the function to the
''tkInt'' stub interface, and I propose calling the function
''TkPhotoGetValidRegion''.
~ Changes to the Tcl-Level API
I propose to add an extra subcommand to photo image instances which
will provide all the access to and manipulation of transparency data.
I propose to call this subcommand ''transparency'' and it will have
subsubcommands to provide access to the various facilities it provides.
Initially I will provide a ''get'' subsubcommand to allow the testing
of the transparency of a single pixel (returned as a boolean value),
and a ''set'' subsubcommand to allow the setting of the transparency of
a single pixel. I anticipate that these will be expanded in the future
to allow the manipulation of transparencies of rectangular regions
(both getting and setting) but I do not supply such at this stage.
| image create photo phImg -file thingy.gif
| if {[phImg transparency get 0 0]} {
| # Top-left pixel is transparent...
| }|| # Toggle transparency...
| phImg transparency set 0 0 [expr {![phImg transparency get 0 0]}]
~ Sample Implementation Patches
Exposing the transparency to C:
http://www.cs.man.ac.uk/~fellowsd/tcl/validRegion.patch
Exposing the transparency to Tcl:
http://www.cs.man.ac.uk/~fellowsd/tcl/patches/transCmd.patch
~ Copyright
This document is placed in the public domain.

# TIP 14: Access to Tk Photo Image Transparency
Author: Donal K. Fellows <fellowsd@cs.man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 22-Nov-2000
Post-History:
Keywords: Tk,photo,transparency,internal,access
Tcl-Version: 8.4.0
-----# Abstract
It is useful for some code \(both extensions and scripts\) to have
access to the transparency information in photo images for various
reasons, but this is not currently available, even via an internal
structure defined in _generic/tkInt.h_. This TIP is aimed at making
the information available, and in such a way at the C level that
backward compatibility is maintained in the future even if the
internal structure definitions change.
# Rationale
I have been working for several years \(on-and-off\) on an extension for
Tk that allows it to have non-rectangular windows
\(<http://www.cs.man.ac.uk/~fellowsd/tcl/shapeidx.html\)> which is an
effect that is great for all sorts of purposes but which comes
particularly into its own when used in conjunction with drag-and-drop
to make drag tokens that obscure only part of what lies underneath
them. However, one of the most useful ways of specifying the shape of
a window turns out to be via images of various kinds, and the natural
way to do this is with the transparency data within the image. The
problem is that this data is locked up entirely within structures that
are completely private to _generic/tkImgPhoto.c_; none of it is
visible at all anywhere else, even within the core. \(There is code
that uses colour data instead to do this sort of trick,
<http://www.sys.uea.ac.uk/~fuzz/tktrans/default.html,> but this is a
slow process and frankly a little strange if we already have
transparency data available.\)
There is also a more general need for scripts to be able to discover
more about the transparent areas of a photo image; at the moment,
there is no access at all to that information at the script level.
# Changes to the C-Level API
To get around this problem, the data member _validRegion_ of the
_PhotoMaster_ structure needs to be made available by some
mechanism. There are two ways of doing this:
1. Placing the _PhotoMaster_ structure, or some version of it, in
_generic/tkInt.h_, or
2. Creating a function to access the data member.
The first way is very cheap initially, but also very inflexible and
creates yet another hidden version dependency \(such as is tackled in
[[5]](5.md)\) should we decide to change the structure for any reason \(we also
have had problems with this sort of thing in the past in relation to
the _Tcl\_Interp_ member _result_, direct access to which has been
deprecated for years, but where there is still existing code that does
it and which forms one of the largest barriers for some extensions
from upgrading to Tcl 8.0 or later.\) It is also unnecessary since
only the core needs to know how to create new instances of the
structure.
The second way, by contrast, is far more flexible in the future as it
will allow us to completely change the internal implementation of
photo image transparency without affecting any extensions at all. The
cost of doing this is that a new entry in one of the stub tables must
be created. Due to the fact that the type of the _validRegion_
member is \(currently\) internal, I propose adding the function to the
_tkInt_ stub interface, and I propose calling the function
_TkPhotoGetValidRegion_.
# Changes to the Tcl-Level API
I propose to add an extra subcommand to photo image instances which
will provide all the access to and manipulation of transparency data.
I propose to call this subcommand _transparency_ and it will have
subsubcommands to provide access to the various facilities it provides.
Initially I will provide a _get_ subsubcommand to allow the testing
of the transparency of a single pixel \(returned as a boolean value\),
and a _set_ subsubcommand to allow the setting of the transparency of
a single pixel. I anticipate that these will be expanded in the future
to allow the manipulation of transparencies of rectangular regions
\(both getting and setting\) but I do not supply such at this stage.
image create photo phImg -file thingy.gif
if {[phImg transparency get 0 0]} {
# Top-left pixel is transparent...
} # Toggle transparency...
phImg transparency set 0 0 [expr {![phImg transparency get 0 0]}]
# Sample Implementation Patches
Exposing the transparency to C:
<http://www.cs.man.ac.uk/~fellowsd/tcl/validRegion.patch>
Exposing the transparency to Tcl:
<http://www.cs.man.ac.uk/~fellowsd/tcl/patches/transCmd.patch># Copyright
This document is placed in the public domain.

TIP: 140Title: Tracing Namespace Modifications
State: Deferred
Type: Project
Tcl-Version: 8.5
Vote: Pending
Post-History:
Version: $Revision: 1.2 $
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Created: 04-Jun-2003
~ Abstract
This TIP allows scripts to register callbacks to monitor alterations
to namespaces, such as their deletion, the creation, renaming and
deletion of commands within the namespace and modifications to the
exports list of the namespace.
~ Rationale
As part of my work on an experimental [112] implementation, I have
discovered that computing the list of public (i.e. exported) commands
from a namespace is a comparatively expensive operation. This means
that it is necessary (well, highly desirable) to cache the list of
exports from a namespace. But then I thought that it might be nice to
expose that interface to use to better enable namespaces to support
other interesting uses, and that it should be possible to implement
the default behaviour in a Tcl script. It is in support of this that
I present this TIP; I also add the ability to trace deletion of the
namespace as a fairly obvious extension.
~ Proposed Change at Tcl Script Level
I propose that the [[trace]] command be extended with a new type of
thing to trace: a namespace. The operations that would trigger a
trace are:
* Namespace deletion (subject to the same sorts of restrictions in
relation to deleted interpreters that are present for traces of
command deletion) with the ops value ''delete''.
* Command list modification (being creation, renaming or deletion of
any command within that namespace) with the ops value ''command''.
Modification due to deletion of the namespace will not trigger
traces with this operation, but instead the ''delete'' operation
outlined above (subject to the caveats relating to interpreter
deletion.)
* Exports list modification (i.e. updates with [[namespace export]])
with the ops value ''export''. Modification due to the deletion of
the namespace will not trigger this traces with this operation, but
instead the ''delete'' operation outlined above (subject to the
caveats relating to interpreter deletion.)
In each case, the trace callback script will have the fully qualified
name of the namespace and the name of the triggering operation
appended to it before execution. In the case of command list
modification by the ''creation'' of a command (which might also be
through import of commands from another namespace, for example) there
will be an extra parameter being the ''local'' name of the command
(without namespace separators.)
These changes will be propagated to each of the [[trace add]], [[trace
remove]] and [[trace info]] subcommands, of course.
~ Proposed Change at C API Level
Yet to be designed, but will be based as closely as possible on the
existing Tcl trace APIs.
~ Copyright
This document has been placed in the public domain.

# TIP 140: Tracing Namespace Modifications
State: Deferred
Type: Project
Tcl-Version: 8.5
Vote: Pending
Post-History:
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Created: 04-Jun-2003
-----# Abstract
This TIP allows scripts to register callbacks to monitor alterations
to namespaces, such as their deletion, the creation, renaming and
deletion of commands within the namespace and modifications to the
exports list of the namespace.
# Rationale
As part of my work on an experimental [[112]](112.md) implementation, I have
discovered that computing the list of public \(i.e. exported\) commands
from a namespace is a comparatively expensive operation. This means
that it is necessary \(well, highly desirable\) to cache the list of
exports from a namespace. But then I thought that it might be nice to
expose that interface to use to better enable namespaces to support
other interesting uses, and that it should be possible to implement
the default behaviour in a Tcl script. It is in support of this that
I present this TIP; I also add the ability to trace deletion of the
namespace as a fairly obvious extension.
# Proposed Change at Tcl Script Level
I propose that the [trace] command be extended with a new type of
thing to trace: a namespace. The operations that would trigger a
trace are:
* Namespace deletion \(subject to the same sorts of restrictions in
relation to deleted interpreters that are present for traces of
command deletion\) with the ops value _delete_.
* Command list modification \(being creation, renaming or deletion of
any command within that namespace\) with the ops value _command_.
Modification due to deletion of the namespace will not trigger
traces with this operation, but instead the _delete_ operation
outlined above \(subject to the caveats relating to interpreter
deletion.\)
* Exports list modification \(i.e. updates with [namespace export]\)
with the ops value _export_. Modification due to the deletion of
the namespace will not trigger this traces with this operation, but
instead the _delete_ operation outlined above \(subject to the
caveats relating to interpreter deletion.\)
In each case, the trace callback script will have the fully qualified
name of the namespace and the name of the triggering operation
appended to it before execution. In the case of command list
modification by the _creation_ of a command \(which might also be
through import of commands from another namespace, for example\) there
will be an extra parameter being the _local_ name of the command
\(without namespace separators.\)
These changes will be propagated to each of the [trace add], [trace
remove] and [trace info] subcommands, of course.
# Proposed Change at C API Level
Yet to be designed, but will be based as closely as possible on the
existing Tcl trace APIs.
# Copyright
This document has been placed in the public domain.

TIP: 141Title: Multiple Initial-Files in [tk_getOpenFile]
Version: $Revision: 1.8 $
Author: David N. Welton <davidw@dedasys.com>
State: Final
Type: Project
Tcl-Version: 8.5
Vote: Done
Created: 18-Jul-2003
Post-History:
~ Abstract
This TIP proposes modifying the semantics of the '''-initialfile'''
option when the '''tk_get*File''' commands are asked to select
multiple files so as to allow several files to be selected initially.
~ Rationale
The '''tk_getOpenFile''' command has a '''-multiple''' option, which
allows multiple files to be selected in the dialog. It also has an
'''-initialfile''' argument. However, at the present time, at least
on Unix, it is impossible to have multiple files selected initially
with '''-initialfile'''. This TIP proposes that '''-initialfile'''take a list of files if '''-multiple''' is also passed as an argument.
As it is possible to select multiple files, it should also be possible
to have multiple files be selected when the widget is created via the
'''-initialfile''' switch.
~~ Additional Notes from Kevin Kenny
~~~ On -initialdir
If the '''-initialdir''' value is not a well-formed path name in its
filesystem (for instance, if a component name contains a null byte or
a character that is not permissible), the behavior is as if
'''-initialdir''' was not specified.
If the object designated by the '''-initialdir''' value does not
exist, or if it is not a directory, nor a symbolic link to one, the
behavior is as if '''-initialdir''' was not specified.
If the '''-initialdir''' value is the empty string, the behavior is as
if '''-initialdir''' was not specified.
~~~ On -initialfile
If '''-multiple''' ''0'' is specified, or '''tk_getSaveFile''' was
called, the '''-initialfile''' value is interpreted as a file name.
If '''-multiple''' ''1'' is specified to '''tk_getOpenFile''', the
'''-initialfile''' value is interpreted as a list of file names. A
list that is not well formed (for example, one containing unbalanced
braces) is not an error, but instead causes the '''-initialfile'''
option to be ignored.
For each file name in the '''-initialfile''' value, the system joins
the directory provided on the '''-initialdir''' option (or the current
working directory if no '''-initialdir''' is supplied) with the
'''-initialfile'''. The resulting path name is normalized as with
'''file normalize''' and then separated into directory and tail
components as with '''file dirname''' and '''file tail'''. Any errors
in this process cause the file name to be ignored.
Once the file name is separated into its components, the directory
part is checked:
* If the '''-initialdir''' option was supplied, and the directory
part of the file name differs from the result of normalizing the
'''-initialdir''' value, the file name is ignored.
* If the '''-initialdir''' option was not supplied, and the directory
part of the file name designates a directory in the file system,
the '''-initialdir''' value is set to that directory. Otherwise,
(since the file cannot exist), the file name is ignored.
* If the given file exists, or if '''tk_getSaveFile was called''',
the tail part of the file name is added to a list of files to
select within the initial directory.
Once '''-initialdir''' and '''-initialfile''' have both been parsed,
the initial directory is known, and the list of initial files is
identified; the files are known to be relative to the initial
directory, and (for '''tk_getOpenFile''') are known to exist.
~ Reference Implementation
The reference implementation exists in a patch
[http://sf.net/tracker/?func=detail&aid=657656&group_id=12997&atid=362997]
which also includes new tests for the Tk test suite and updated
documentation.
~ Copyright
This document is in the public domain.

# TIP 141: Multiple Initial-Files in [tk_getOpenFile]
Author: David N. Welton <davidw@dedasys.com>
State: Final
Type: Project
Tcl-Version: 8.5
Vote: Done
Created: 18-Jul-2003
Post-History:
-----# Abstract
This TIP proposes modifying the semantics of the **-initialfile**
option when the **tk\_get\*File** commands are asked to select
multiple files so as to allow several files to be selected initially.
# Rationale
The **tk\_getOpenFile** command has a **-multiple** option, which
allows multiple files to be selected in the dialog. It also has an
**-initialfile** argument. However, at the present time, at least
on Unix, it is impossible to have multiple files selected initially
with **-initialfile**. This TIP proposes that **-initialfile**take a list of files if **-multiple** is also passed as an argument.
As it is possible to select multiple files, it should also be possible
to have multiple files be selected when the widget is created via the
**-initialfile** switch.
## Additional Notes from Kevin Kenny
### On -initialdir
If the **-initialdir** value is not a well-formed path name in its
filesystem \(for instance, if a component name contains a null byte or
a character that is not permissible\), the behavior is as if
**-initialdir** was not specified.
If the object designated by the **-initialdir** value does not
exist, or if it is not a directory, nor a symbolic link to one, the
behavior is as if **-initialdir** was not specified.
If the **-initialdir** value is the empty string, the behavior is as
if **-initialdir** was not specified.
### On -initialfile
If **-multiple** _0_ is specified, or **tk\_getSaveFile** was
called, the **-initialfile** value is interpreted as a file name.
If **-multiple** _1_ is specified to **tk\_getOpenFile**, the
**-initialfile** value is interpreted as a list of file names. A
list that is not well formed \(for example, one containing unbalanced
braces\) is not an error, but instead causes the **-initialfile**
option to be ignored.
For each file name in the **-initialfile** value, the system joins
the directory provided on the **-initialdir** option \(or the current
working directory if no **-initialdir** is supplied\) with the
**-initialfile**. The resulting path name is normalized as with
**file normalize** and then separated into directory and tail
components as with **file dirname** and **file tail**. Any errors
in this process cause the file name to be ignored.
Once the file name is separated into its components, the directory
part is checked:
* If the **-initialdir** option was supplied, and the directory
part of the file name differs from the result of normalizing the
**-initialdir** value, the file name is ignored.
* If the **-initialdir** option was not supplied, and the directory
part of the file name designates a directory in the file system,
the **-initialdir** value is set to that directory. Otherwise,
\(since the file cannot exist\), the file name is ignored.
* If the given file exists, or if **tk\_getSaveFile was called**,
the tail part of the file name is added to a list of files to
select within the initial directory.
Once **-initialdir** and **-initialfile** have both been parsed,
the initial directory is known, and the list of initial files is
identified; the files are known to be relative to the initial
directory, and \(for **tk\_getOpenFile**\) are known to exist.
# Reference Implementation
The reference implementation exists in a patch
<http://sf.net/tracker/?func=detail&aid=657656&group_id=12997&atid=362997>
which also includes new tests for the Tk test suite and updated
documentation.
# Copyright
This document is in the public domain.

TIP: 142Title: Search Path Variable to Lookup Command Names in Namespaces
Version: $Revision: 1.3 $
Author: Ulrich Schoebel <ulrich.schoebel@sigos.de>
Tcl-Version: 8.5
State: Withdrawn
Type: Project
Vote: Pending
Created: 23-Jul-2003
Post-History:
Keywords: namespace, command lookup, search path
~ Abstract
This TIP adds a Tcl variable to define the search path for command
name lookup across namespaces.
~ Rationale
Command names (as well as variable names) are currently looked up
first in the current namspace, then, if not found, in the global
namespace.
It is often very useful to hide the commands defined in a subnamespace
from being visible from upper namespaces by ''info commands
namespace::*''. On the other hand, programmers want to use these
commands without having to type a qualified name.
* Example:
| namespace eval ns1 {
| proc p1 {} {
| puts "[p2]"
| }| }|| namespace eval ns1::ns2 {
| proc p2 {} {
| return hello
| }| }
Evaluation of ''ns1::p1'' would currently lead to an error, because
''p2'' could not be found. Even worse, if a procedure ''p2'' exists
in the global namespace, the wrong procedure would be evaluated.
~ Proposal
Add a variable ''tcl_namespacePath'' or, to avoid confusion with
variables containing file system paths, ''tcl_namespaceSearch'', that
contains a list of namespaces to be searched in that order.
The default value would be ''[[list [[namespace current]] ::]]''.
In the above example ''tcl_namespacePath'' would be set to ''[[list
[[namespace current]] [[namespace current]]::ns2]]''. ''p2'' would be
found and not unintentionally be substituted by ''::p2''.
~ Alternative
For ease of implementation and, maybe, for programmers convenience it
might be useful to always prepend the contents of this variable with
''[[namespace current]]''. The programmer expects a certain
"automatism" for this component of the search path.
Then the default value would be ''::''.
~ Implementation
To be done when this TIP is accepted.
~ Notice of Withdrawal
This TIP was Withdrawn by the TIP Editor following discussion on the
tcl-core mailing list. The following is a summary of reasons for
withdrawal:
> Insufficiently subtle. 52 will break any code that assumes the
current behaviour (and you can bet someone will have that
assumption) and 142 doesn't let two namespaces have different
search paths (unless the variable is always interpreted locally,
which just creates bizarre variable name magic.)
~ Copyright
This document is placed in the public domain.

# TIP 142: Search Path Variable to Lookup Command Names in Namespaces
Author: Ulrich Schoebel <ulrich.schoebel@sigos.de>
Tcl-Version: 8.5
State: Withdrawn
Type: Project
Vote: Pending
Created: 23-Jul-2003
Post-History:
Keywords: namespace, command lookup, search path
-----# Abstract
This TIP adds a Tcl variable to define the search path for command
name lookup across namespaces.
# Rationale
Command names \(as well as variable names\) are currently looked up
first in the current namspace, then, if not found, in the global
namespace.
It is often very useful to hide the commands defined in a subnamespace
from being visible from upper namespaces by _info commands
namespace::\*_. On the other hand, programmers want to use these
commands without having to type a qualified name.
* Example:
namespace eval ns1 {
proc p1 {} {
puts "[p2]"
} } namespace eval ns1::ns2 {
proc p2 {} {
return hello
} }
Evaluation of _ns1::p1_ would currently lead to an error, because
_p2_ could not be found. Even worse, if a procedure _p2_ exists
in the global namespace, the wrong procedure would be evaluated.
# Proposal
Add a variable _tcl\_namespacePath_ or, to avoid confusion with
variables containing file system paths, _tcl\_namespaceSearch_, that
contains a list of namespaces to be searched in that order.
The default value would be _[list [namespace current] ::]_.
In the above example _tcl\_namespacePath_ would be set to _[list
[namespace current] [namespace current]::ns2]_. _p2_ would be
found and not unintentionally be substituted by _::p2_.
# Alternative
For ease of implementation and, maybe, for programmers convenience it
might be useful to always prepend the contents of this variable with
_[namespace current]_. The programmer expects a certain
"automatism" for this component of the search path.
Then the default value would be _::_.
# Implementation
To be done when this TIP is accepted.
# Notice of Withdrawal
This TIP was Withdrawn by the TIP Editor following discussion on the
tcl-core mailing list. The following is a summary of reasons for
withdrawal:
> Insufficiently subtle. 52 will break any code that assumes the
current behaviour \(and you can bet someone will have that
assumption\) and 142 doesn't let two namespaces have different
search paths \(unless the variable is always interpreted locally,
which just creates bizarre variable name magic.\)
# Copyright
This document is placed in the public domain.

TIP: 143Title: An Interpreter Resource Limiting Framework
Version: $Revision: 1.13 $
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 25-Jul-2003
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP introduces a mechanism for creating and manipulating
per-interpreter resource limits. This stops several significant
classes of denial-of-service attack, and can also be used to do things
like guaranteeing an answer within a particular amount of time.
~Rationale
Currently it is trivial for scripts running in even safe Tcl
interpreters to conduct a denial-of-service attack on the thread in
which they are running. All they have to do is write a simple
infinite loop with the '''for''' or '''while''' commands. Of course, it
is possible to put a stop to this by hiding those commands from the
interpreter, but even then it is possible to simulate the DoS effect
with the help of a procedure like this (which even under ideal
conditions will take over three days to run):
|proc x s {eval $s; eval $s; eval $s; eval $s}
|x {x {x {x {x {x {x {x {x {x {x {x {x {x {sleep 1}}}}}}}}}}}}}}
The easiest way around this, of course, is the resource quota as used
by heavyweight operating systems, perhaps combined with the
''alarm(2)'' system call. Or at least that would be the way to do it
if it wasn't for the fact that those are ridiculously heavy
sledgehammers to take to this particular nut. Luckily we control the
execution environment - it is a Tcl interpreter after all - so we can
implement our own checks. It is this that is the aim of this TIP.
~~Efficiency
Efficiency of any resource monitoring system is naturally a major
concern; it is relatively simple to create a resource monitoring system
but quite a lot harder to arrange for that monitoring to be cheap enough
that its use does not greatly impact on the general speed of the program
(in this case, Tcl.) The costs of checking time limits in particular
can be somewhat excessive because of the necessity of performing a
system call to carry out the check, but if you're performing any action
a lot, it can get costly.
My strategy for limiting the impact upon performance is to provide a
programmer-tunable per-limit parameter called the granularity. This
specifies how often (out of all the locations in the processing of the
Tcl interpreter and bytecode engine) the limits should be checked; the
granularity is per-limit because there is this distinct difference in
the cost of checking different kinds of limits, and the limits are
tunable because the ideal frequency depends very largely on the code
being limited. This can be combined with the fact that the test to see
if ''any'' limits have been turned on can be made extremely cheap (i.e.
just a comparison of a memory location to zero) and it allows unlimited
interpreters to perform at almost the same speed as before.
~A Tcl API for Limit Control
I propose to add a subcommand '''limit''' to the '''interp''' command and
to the object-like interpreter access commands. It will be an error
for any safe interpreter to call the '''limit''' subcommand; master
interpreters may enable it for their safe slaves via the mechanism
interpreter aliases, as is normal for security policies.
The first argument to the '''interp limit''' command will be the name of
the interpreter whose limit is to be inspected. The second argument
will be the name of the limit being modified; this TIP defines two
kinds of limits:
time: Specifies when the interpreter will be prohibited from
performing further executions. The limit is not guaranteed to be
hit at exactly the time given, but will not be hit before then.
command: Specifies a maximum number of commands (see '''info
cmdcount''') to be executed.
The third and subsequent arguments specify a series of properties
(each of which has a name that begins with a hyphen) which may be set
(if there are pairs of arguments, being the property name and the
value to set it to) or read (if there is just the property name on its
own). If no properties are listed at all, all dictionary (see [111])
of all properties for the particular limit is returned. The set of
properties is limit-specific, but always includes the following:
-command: A Tcl script to be executed in the global namespace of the
interpreter reading/writing the property when the limit is found to
be exceeded in the limited interpreter. If no command callback is
defined ''for this interpreter'', the '''-command''' option will be
the empty string. Note that multiple interpreters may set command
callbacks that are distinct from each other; an interpreter may not
see what other interpreters have installed (except by running a
script in those foreign interpreters, of course.) The order of
calling of the callbacks is not defined by this TIP.
> Where a callback returns any kind of exceptional condition
(i.e. the result isn't TCL_OK) a background error is flagged up.
-granularity: Limits will always be checked in locations where the
''Tcl_Async*()'' API is called, as this is called regularly by the Tcl
interpreter (with a few exceptions relating to event loop
handling.) However, the cost of just checking a limit can be
quite appreciable (it might involve system calls, say) so this
property allows the control of how frequently, out of the
opportunities to check a limit, are such limits actually checked.
If the granularity is ''1'', the limit will be checked
every time. It is an error to try to set the granularity to less
than ''1''.
When an interpreter hits a limit, it first runs all command callbacks
in their respective interpreters. Once that is done, the interpreter
rechecks the limit (since the command callbacks might have decided to
raise or remove it) and if it is still exceeded it bails out the
interpreter in the following way:
* A flag is set in the interpreter to mark the interpreter as having
exceeded its limits.
* The currently executing command/script in the interpreter is made
to return with code TCL_ERROR. (This is superior to using a novel
return code, as third-party extensions are usually far better at
handling error cases!)
* The '''catch''' command will only catch errors if the interpreter
containing it does not have the flag mentioned just above set.
Similarly, further trips round the internal loops of the
'''vwait''', '''update''' and '''tkwait''' commands will not
proceed with that flag set. (Extensions can find this information
out by using ''Tcl_LimitExceeded()''; see below.) No calls to
'''bgerror''' should be made.
* Once the execution unwinds out of the interpreter so that no
further invocations of the interpreter are left on the call-stack,
the flag is reset. The same is true if the limits are adjusted in
any way. Note that attempting to execute things within the
interpreter without raising the limits will result in the limit
being hit immediately.
When resource limits are being used, unlimited master interpreters
should take care to use the '''catch''' command when calling their
limited slaves. Otherwise hitting the limit in the slave might well
smash the master as well, just because of general error propagation.
But that is good practise anyway.
~~Time Limits
Time limits are specified in terms of the time when the limit will be
hit. Setting the limit to the current time ensures that the limit
will be immediately activated. Time limits have two options for
specifying the limit.
-seconds: This sets the absolute time (in seconds from the epoch, as
returned by '''clock seconds''') that the time limit is hit. If
set to the empty string, the limit is removed. If no time limit is
set, this option is empty when inspected.
-milliseconds: This sets the number of milliseconds after the start
of the second specified in the '''-seconds''' option that the time
limit is hit. May only be set to the empty string if the
'''-seconds''' option is empty and present, and may only be set to
a numeric value if the '''-seconds''' option is unspecified or
present and non-empty (it is always safe to leave this option
unspecified.) If no time limit is set, this option is empty when
inspected.
Where a time-limited interpreter creates a slave interpreter, the
slave will get the same time-limit as the creating master interpreter.
~~Command Limits
Command limits are specified in terms of the number of commands (see
'''info cmdcount''' for a definition of the metric) that may be
executed before the limit is hit. Command limits have the following
extra option for specifying and inspecting the limit.
-value: The number of commands (integer of course) that may actually
be executed. If set to the empty string, the limit is removed, and
when introspecting, unlimited interpreters return empty strings for
this value.
Where a command-limited interpreter creates a slave interpreter, the
slave will get the command-limit ''0'' after initialisation
(i.e. after the return from the call to ''Tcl_CreateSlave()'') and
will be unable to execute and commands until the limit for the slave
is raised. Master interpreters implementing security policies for
safe interpreters might want to set such limits semi-automatically to
something more useful by deducting command-executions from the
creating interpreter to its new slave.
~A C-level API for Limit Control
It is also desirable for there to be a general C API for controlling
resource limits. Not only does this provide control to extension
authors that can't be easily smashed by Tcl scripts by accident, but
it also makes implementation of the Tcl API to the limit subsystem
easier to create as well.
* int '''Tcl_LimitReady'''(Tcl_Interp *''interp'')
> Test whether a limit is ready to be checked according to its
granularity rules. Result is boolean. Note that this is a cheap
call.
* int '''Tcl_LimitCheck'''(Tcl_Interp *''interp'')
> Test whether a limit has been exceeded. Result is boolean. Note
that this call is potentially expensive (checking a time limit
requires a minimum of one system call.)
* int '''Tcl_LimitExceeded'''(Tcl_Interp *''interp'')
> Test whether the interpreter is in a state where a limit has been
previously exceeded (i.e. whether a previous call to
'''Tcl_LimitCheck''' had indicated that a limit had been hit) and
not yet reset (by extending or removing the relevant limit.)
Result is boolean. Note that this is a cheap call.
* int '''Tcl_LimitTypeEnabled'''(Tcl_Interp *''interp'', int ''type'')
> Test whether the given type of limit is turned on. Result is boolean.
Note that this is a cheap call.
* int '''Tcl_LimitTypeExceeded'''(Tcl_Interp *''interp'', int ''type'')
> Test whether the given type of limit has been hit (in the notion of
'''Tcl_LimitExceeded''') and not yet reset. Result is boolean.
Note that this is a cheap call.
* void '''Tcl_LimitTypeSet'''(Tcl_Interp *''interp'', int ''type'')
> Turn on the given type of limit.
* void '''Tcl_LimitTypeReset'''(Tcl_Interp *''interp'', int ''type'')
> Turn off the given type of limit.
* void '''Tcl_LimitAddHandler'''(Tcl_Interp *''interp'', int ''type'',
Tcl_LimitHandlerProc *''handlerProc'', ClientData ''clientData'',
Tcl_LimitHandlerDeleteProc *''deleteProc'')
> Add a limit handler for the given type of limit and specify some
caller context and a scheme for deleting that context.
* void '''Tcl_LimitRemoveHandler'''(Tcl_Interp *''interp'', int ''type'',
Tcl_LimitHandlerProc *''handlerProc'', ClientData ''clientData'')
> Remove a limit handler for the given type of limit. It is not an
error to delete a limit that was not set; in that case, nothing is
changed.
* void '''Tcl_LimitSetCommands'''(Tcl_Interp *''interp'',
int ''commandLimit'')
> Set a limiting value on the number of commands, and reset whether
the limit has been triggered. Does not enable the limit.
* int '''Tcl_LimitGetCommands'''(Tcl_Interp *''interp'')
> Get the current limit on the number of commands.
* void '''Tcl_LimitSetTime'''(Tcl_Interp *''interp'', Tcl_Time *''timeLimitPtr'')
> Set a limiting value (copying it from the buffer pointed to by
''timeLimitPtr'') on the latest time that the code may execute, and
reset whether the limit has been triggered. Does not enable the
limit. The ''usec'' field of the buffer should be in the range 0
to 999999.
* void '''Tcl_LimitGetTime'''(Tcl_Interp *''interp'', Tcl_Time *''timeLimitPtr'')
> Get the current limit on the execution time, writing it into the
buffer pointed to by ''timeLimitPtr''.
* void '''Tcl_LimitSetGranularity'''(Tcl_Interp *''interp'', int ''type'',
int ''granularity'')
> Set the checking granularity for the given type of limit.
* int '''Tcl_LimitGetGranularity'''(Tcl_Interp *''interp'', int ''type'')
> Get the checking granularity for the given type of limit.
Above, ''type'' is either TCL_LIMIT_COMMANDS or TCL_LIMIT_TIME,
''handlerProc'' is a pointer to a function that takes two parameters
(a ClientData and a Tcl_Interp*) and returns void, and ''deleteProc''
is a pointer to a function that takes a single parameter (a
ClientData) and returns void. The key is that ''handlerProc''s are
called when a limit is hit (they are used to implement the guts of the
'''-command''' option), and when the callback is deleted for any
reason (including a call to '''Tcl_LimitRemoveHandler''' and deleting
the limited interpreter) the ''deleteProc'' is called to release the
resources consumed by the ''clientData'' context.
~Use Cases
~~WebServices
One use for this sort of code might be in a web-services context where
it is important to return a message to some client code within some
interval. Using an in-process limiting mechanism allows this to be
implemented in a far more light-weight fashion, as the alternative would
be to fork off a new small application server for each incoming request
and it would be considerably more complex to have a scripted executive
that decides (possibly by examining the stack) whether a failure to
deliver an answer within bounds is serious, or whether some extra
resources should be granted to allow execution to run to completion.
Other high-performance server applications would also be likely to gain
from this sort of thing.
~~Profiling
It is possible to use the limiting code (and especially the script
callbacks) to write a Tcl profiler. Every time the limit runs out, the
callback can examine the Tcl stack in the limited interpreter and then
assign some more resources to last up until the next profile trap.
~~Untrusted Code Execution
As indicated earlier, these limits can be used to increase control over
untrusted code running in safe interpreters. While it would be
necessary to extend this to memory consumption for every aspect that
could be impacted by some malicious code to be controllable, having
control over the number of commands that may be executed and how long
those commands may take gives a much higher degree of control than
currently exists, and is thus a monotonic improvement.
~Possible Future Directions
There are some obvious other things that could be brought within this
framework, but which I've left out for various reasons:
call stack: We already do limiting of this, so bring that within this
framework would be a nice regularisation. On the other hand, such
a change would not be backward-compatible, and it might not be safe
to perform the callbacks either (especially as overflowing the
stack is fatal in a way that overrunning on time is not.)
memory: Limiting the amount of memory that an interpreter may
allocate for its own use would be very nice. But conceptually
difficult to do (what about memory shared between interpreters?),
expensive to keep track of (memory debugging is known to add a lot
of overhead, and memory limiting would be more intrusive) and a
really big change technically too (i.e. you'd be rewriting a
significant fraction of the Tcl C API to make sure that every
memory allocation knows which interpreter owns it.)
open channels, file sizes, etc.: Most other things can be limited in
Tcl right now without special support.
~Implementation
An example implementation of this TIP is available as a patch
[http://sf.net/tracker/?func=detail&aid=926771&group_id=10894&atid=310894].
~Copyright
This document is placed in the public domain.

# TIP 143: An Interpreter Resource Limiting Framework
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 25-Jul-2003
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP introduces a mechanism for creating and manipulating
per-interpreter resource limits. This stops several significant
classes of denial-of-service attack, and can also be used to do things
like guaranteeing an answer within a particular amount of time.
# Rationale
Currently it is trivial for scripts running in even safe Tcl
interpreters to conduct a denial-of-service attack on the thread in
which they are running. All they have to do is write a simple
infinite loop with the **for** or **while** commands. Of course, it
is possible to put a stop to this by hiding those commands from the
interpreter, but even then it is possible to simulate the DoS effect
with the help of a procedure like this \(which even under ideal
conditions will take over three days to run\):
proc x s {eval $s; eval $s; eval $s; eval $s}
x {x {x {x {x {x {x {x {x {x {x {x {x {x {sleep 1}}}}}}}}}}}}}}
The easiest way around this, of course, is the resource quota as used
by heavyweight operating systems, perhaps combined with the
_alarm\(2\)_ system call. Or at least that would be the way to do it
if it wasn't for the fact that those are ridiculously heavy
sledgehammers to take to this particular nut. Luckily we control the
execution environment - it is a Tcl interpreter after all - so we can
implement our own checks. It is this that is the aim of this TIP.
## Efficiency
Efficiency of any resource monitoring system is naturally a major
concern; it is relatively simple to create a resource monitoring system
but quite a lot harder to arrange for that monitoring to be cheap enough
that its use does not greatly impact on the general speed of the program
\(in this case, Tcl.\) The costs of checking time limits in particular
can be somewhat excessive because of the necessity of performing a
system call to carry out the check, but if you're performing any action
a lot, it can get costly.
My strategy for limiting the impact upon performance is to provide a
programmer-tunable per-limit parameter called the granularity. This
specifies how often \(out of all the locations in the processing of the
Tcl interpreter and bytecode engine\) the limits should be checked; the
granularity is per-limit because there is this distinct difference in
the cost of checking different kinds of limits, and the limits are
tunable because the ideal frequency depends very largely on the code
being limited. This can be combined with the fact that the test to see
if _any_ limits have been turned on can be made extremely cheap \(i.e.
just a comparison of a memory location to zero\) and it allows unlimited
interpreters to perform at almost the same speed as before.
# A Tcl API for Limit Control
I propose to add a subcommand **limit** to the **interp** command and
to the object-like interpreter access commands. It will be an error
for any safe interpreter to call the **limit** subcommand; master
interpreters may enable it for their safe slaves via the mechanism
interpreter aliases, as is normal for security policies.
The first argument to the **interp limit** command will be the name of
the interpreter whose limit is to be inspected. The second argument
will be the name of the limit being modified; this TIP defines two
kinds of limits:
time: Specifies when the interpreter will be prohibited from
performing further executions. The limit is not guaranteed to be
hit at exactly the time given, but will not be hit before then.
command: Specifies a maximum number of commands \(see **info
cmdcount**\) to be executed.
The third and subsequent arguments specify a series of properties
\(each of which has a name that begins with a hyphen\) which may be set
\(if there are pairs of arguments, being the property name and the
value to set it to\) or read \(if there is just the property name on its
own\). If no properties are listed at all, all dictionary \(see [[111]](111.md)\)
of all properties for the particular limit is returned. The set of
properties is limit-specific, but always includes the following:
-command: A Tcl script to be executed in the global namespace of the
interpreter reading/writing the property when the limit is found to
be exceeded in the limited interpreter. If no command callback is
defined _for this interpreter_, the **-command** option will be
the empty string. Note that multiple interpreters may set command
callbacks that are distinct from each other; an interpreter may not
see what other interpreters have installed \(except by running a
script in those foreign interpreters, of course.\) The order of
calling of the callbacks is not defined by this TIP.
> Where a callback returns any kind of exceptional condition
\(i.e. the result isn't TCL\_OK\) a background error is flagged up.
-granularity: Limits will always be checked in locations where the
_Tcl\_Async\*\(\)_ API is called, as this is called regularly by the Tcl
interpreter \(with a few exceptions relating to event loop
handling.\) However, the cost of just checking a limit can be
quite appreciable \(it might involve system calls, say\) so this
property allows the control of how frequently, out of the
opportunities to check a limit, are such limits actually checked.
If the granularity is _1_, the limit will be checked
every time. It is an error to try to set the granularity to less
than _1_.
When an interpreter hits a limit, it first runs all command callbacks
in their respective interpreters. Once that is done, the interpreter
rechecks the limit \(since the command callbacks might have decided to
raise or remove it\) and if it is still exceeded it bails out the
interpreter in the following way:
* A flag is set in the interpreter to mark the interpreter as having
exceeded its limits.
* The currently executing command/script in the interpreter is made
to return with code TCL\_ERROR. \(This is superior to using a novel
return code, as third-party extensions are usually far better at
handling error cases!\)
* The **catch** command will only catch errors if the interpreter
containing it does not have the flag mentioned just above set.
Similarly, further trips round the internal loops of the
**vwait**, **update** and **tkwait** commands will not
proceed with that flag set. \(Extensions can find this information
out by using _Tcl\_LimitExceeded\(\)_; see below.\) No calls to
**bgerror** should be made.
* Once the execution unwinds out of the interpreter so that no
further invocations of the interpreter are left on the call-stack,
the flag is reset. The same is true if the limits are adjusted in
any way. Note that attempting to execute things within the
interpreter without raising the limits will result in the limit
being hit immediately.
When resource limits are being used, unlimited master interpreters
should take care to use the **catch** command when calling their
limited slaves. Otherwise hitting the limit in the slave might well
smash the master as well, just because of general error propagation.
But that is good practise anyway.
## Time Limits
Time limits are specified in terms of the time when the limit will be
hit. Setting the limit to the current time ensures that the limit
will be immediately activated. Time limits have two options for
specifying the limit.
-seconds: This sets the absolute time \(in seconds from the epoch, as
returned by **clock seconds**\) that the time limit is hit. If
set to the empty string, the limit is removed. If no time limit is
set, this option is empty when inspected.
-milliseconds: This sets the number of milliseconds after the start
of the second specified in the **-seconds** option that the time
limit is hit. May only be set to the empty string if the
**-seconds** option is empty and present, and may only be set to
a numeric value if the **-seconds** option is unspecified or
present and non-empty \(it is always safe to leave this option
unspecified.\) If no time limit is set, this option is empty when
inspected.
Where a time-limited interpreter creates a slave interpreter, the
slave will get the same time-limit as the creating master interpreter.
## Command Limits
Command limits are specified in terms of the number of commands \(see
**info cmdcount** for a definition of the metric\) that may be
executed before the limit is hit. Command limits have the following
extra option for specifying and inspecting the limit.
-value: The number of commands \(integer of course\) that may actually
be executed. If set to the empty string, the limit is removed, and
when introspecting, unlimited interpreters return empty strings for
this value.
Where a command-limited interpreter creates a slave interpreter, the
slave will get the command-limit _0_ after initialisation
\(i.e. after the return from the call to _Tcl\_CreateSlave\(\)_\) and
will be unable to execute and commands until the limit for the slave
is raised. Master interpreters implementing security policies for
safe interpreters might want to set such limits semi-automatically to
something more useful by deducting command-executions from the
creating interpreter to its new slave.
# A C-level API for Limit Control
It is also desirable for there to be a general C API for controlling
resource limits. Not only does this provide control to extension
authors that can't be easily smashed by Tcl scripts by accident, but
it also makes implementation of the Tcl API to the limit subsystem
easier to create as well.
* int **Tcl\_LimitReady**\(Tcl\_Interp \*_interp_\)
> Test whether a limit is ready to be checked according to its
granularity rules. Result is boolean. Note that this is a cheap
call.
* int **Tcl\_LimitCheck**\(Tcl\_Interp \*_interp_\)
> Test whether a limit has been exceeded. Result is boolean. Note
that this call is potentially expensive \(checking a time limit
requires a minimum of one system call.\)
* int **Tcl\_LimitExceeded**\(Tcl\_Interp \*_interp_\)
> Test whether the interpreter is in a state where a limit has been
previously exceeded \(i.e. whether a previous call to
**Tcl\_LimitCheck** had indicated that a limit had been hit\) and
not yet reset \(by extending or removing the relevant limit.\)
Result is boolean. Note that this is a cheap call.
* int **Tcl\_LimitTypeEnabled**\(Tcl\_Interp \*_interp_, int _type_\)
> Test whether the given type of limit is turned on. Result is boolean.
Note that this is a cheap call.
* int **Tcl\_LimitTypeExceeded**\(Tcl\_Interp \*_interp_, int _type_\)
> Test whether the given type of limit has been hit \(in the notion of
**Tcl\_LimitExceeded**\) and not yet reset. Result is boolean.
Note that this is a cheap call.
* void **Tcl\_LimitTypeSet**\(Tcl\_Interp \*_interp_, int _type_\)
> Turn on the given type of limit.
* void **Tcl\_LimitTypeReset**\(Tcl\_Interp \*_interp_, int _type_\)
> Turn off the given type of limit.
* void **Tcl\_LimitAddHandler**\(Tcl\_Interp \*_interp_, int _type_,
Tcl\_LimitHandlerProc \*_handlerProc_, ClientData _clientData_,
Tcl\_LimitHandlerDeleteProc \*_deleteProc_\)
> Add a limit handler for the given type of limit and specify some
caller context and a scheme for deleting that context.
* void **Tcl\_LimitRemoveHandler**\(Tcl\_Interp \*_interp_, int _type_,
Tcl\_LimitHandlerProc \*_handlerProc_, ClientData _clientData_\)
> Remove a limit handler for the given type of limit. It is not an
error to delete a limit that was not set; in that case, nothing is
changed.
* void **Tcl\_LimitSetCommands**\(Tcl\_Interp \*_interp_,
int _commandLimit_\)
> Set a limiting value on the number of commands, and reset whether
the limit has been triggered. Does not enable the limit.
* int **Tcl\_LimitGetCommands**\(Tcl\_Interp \*_interp_\)
> Get the current limit on the number of commands.
* void **Tcl\_LimitSetTime**\(Tcl\_Interp \*_interp_, Tcl\_Time \*_timeLimitPtr_\)
> Set a limiting value \(copying it from the buffer pointed to by
_timeLimitPtr_\) on the latest time that the code may execute, and
reset whether the limit has been triggered. Does not enable the
limit. The _usec_ field of the buffer should be in the range 0
to 999999.
* void **Tcl\_LimitGetTime**\(Tcl\_Interp \*_interp_, Tcl\_Time \*_timeLimitPtr_\)
> Get the current limit on the execution time, writing it into the
buffer pointed to by _timeLimitPtr_.
* void **Tcl\_LimitSetGranularity**\(Tcl\_Interp \*_interp_, int _type_,
int _granularity_\)
> Set the checking granularity for the given type of limit.
* int **Tcl\_LimitGetGranularity**\(Tcl\_Interp \*_interp_, int _type_\)
> Get the checking granularity for the given type of limit.
Above, _type_ is either TCL\_LIMIT\_COMMANDS or TCL\_LIMIT\_TIME,
_handlerProc_ is a pointer to a function that takes two parameters
\(a ClientData and a Tcl\_Interp\*\) and returns void, and _deleteProc_
is a pointer to a function that takes a single parameter \(a
ClientData\) and returns void. The key is that _handlerProc_s are
called when a limit is hit \(they are used to implement the guts of the
**-command** option\), and when the callback is deleted for any
reason \(including a call to **Tcl\_LimitRemoveHandler** and deleting
the limited interpreter\) the _deleteProc_ is called to release the
resources consumed by the _clientData_ context.
# Use Cases
## WebServices
One use for this sort of code might be in a web-services context where
it is important to return a message to some client code within some
interval. Using an in-process limiting mechanism allows this to be
implemented in a far more light-weight fashion, as the alternative would
be to fork off a new small application server for each incoming request
and it would be considerably more complex to have a scripted executive
that decides \(possibly by examining the stack\) whether a failure to
deliver an answer within bounds is serious, or whether some extra
resources should be granted to allow execution to run to completion.
Other high-performance server applications would also be likely to gain
from this sort of thing.
## Profiling
It is possible to use the limiting code \(and especially the script
callbacks\) to write a Tcl profiler. Every time the limit runs out, the
callback can examine the Tcl stack in the limited interpreter and then
assign some more resources to last up until the next profile trap.
## Untrusted Code Execution
As indicated earlier, these limits can be used to increase control over
untrusted code running in safe interpreters. While it would be
necessary to extend this to memory consumption for every aspect that
could be impacted by some malicious code to be controllable, having
control over the number of commands that may be executed and how long
those commands may take gives a much higher degree of control than
currently exists, and is thus a monotonic improvement.
# Possible Future Directions
There are some obvious other things that could be brought within this
framework, but which I've left out for various reasons:
call stack: We already do limiting of this, so bring that within this
framework would be a nice regularisation. On the other hand, such
a change would not be backward-compatible, and it might not be safe
to perform the callbacks either \(especially as overflowing the
stack is fatal in a way that overrunning on time is not.\)
memory: Limiting the amount of memory that an interpreter may
allocate for its own use would be very nice. But conceptually
difficult to do \(what about memory shared between interpreters?\),
expensive to keep track of \(memory debugging is known to add a lot
of overhead, and memory limiting would be more intrusive\) and a
really big change technically too \(i.e. you'd be rewriting a
significant fraction of the Tcl C API to make sure that every
memory allocation knows which interpreter owns it.\)
open channels, file sizes, etc.: Most other things can be limited in
Tcl right now without special support.
# Implementation
An example implementation of this TIP is available as a patch
<http://sf.net/tracker/?func=detail&aid=926771&group_id=10894&atid=310894> .
# Copyright
This document is placed in the public domain.

TIP: 144Title: Argument Expansion Syntax
Version: $Revision: 1.8 $
Author: Peter Spjuth <peter.spjuth@space.se>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Author: <dgp@users.sf.net>
State: Withdrawn
Type: Project
Vote: Pending
Created: 26-Jul-2003
Post-History:
Obsoleted-By: 157
Tcl-Version: 8.5
~ Abstract
This TIP proposes to add syntax in Tcl to perform argument expansion
in a safe and efficient manner.
~ Introduction
Many commands take a variable number of arguments and often you find
yourself with those arguments in a list. This list must then be
expanded into individual arguments to the command. This is currently
done with eval:
|eval destroy [winfo children .]
This is a bit obscure and also very error prone when the command
becomes more complex. It is also inefficient and not object safe, why
something specialised in doing this would be better.
~ Background
See also [103]. Please also see a summary of a
poll of TCLCORE readers taken after [103] was
rejected.
http://wiki.tcl.tk/9462
~ Rationale
For examples three statements are used. This is the eval version:
|eval destroy [winfo children .]
|eval button .b $stdargs -text \$mytext -bd $border
|eval exec \$prog $opts1 [getMoreopts] \$file1 \$file2
The eval version would be even more complex if the lists that are to
be expanded are not known to be pure. To be really safe the last
would be:
|eval exec \$prog [lrange $opts1 0 end] [lrange [getMoreopts] 0 end] \$file1 \$file2
With the proposed syntax they become:
|destroy {}[winfo children .]
|button .b {}$stdargs -text $mytext -bd $border
|exec $prog {}$opts1 {}[getMoreopts] $file1 $file2
The advantage of using syntax for this is that the command do not get
obscured. In the examples destroy/button/exec is the most important
information on each line and it gets to be first on the line.
~ Specification
If a word starts with a pair of braces, "{}", and is followed by a non
whitespace character it signifies argument expansion. The braces are
removed and the rest of the word is parsed and substituted as any other
word. The
character after the removed "{}" counts as a first character in the
rules about open braces and double quotes. After substitution, the
word is then parsed as
a list (as if with ''Tcl_SplitList()'' or ''Tcl_GetListFromObj'') and
each element of the list is added to the command being built as a
separate word with no further parsing.
Before executing the command any word to be expanded is treated as a
list where each element becomes one separate argument to the command.
''Note 1:'' A word should really start with {} to trigger expansion
which means that words like these are not expanded:
|cmd "{}$temp" \{}[something]
''Note 2:'' Expansion is typically most useful with words like:
|cmd {}$var {}[somecmd $arg] {}$arr([cmd $arg])
But things like this are also legal:
|cmd {}word {}$x,$y {}[foo]xy[apa] {}{apa bepa}
~ Motivating Examples
Many of the examples that make this TIP the way it is come from either
advanced usage of commands like [[exec]] or from Tk.
Consider the case where you have lists of options for several external
commands that are to be executed together in a pipeline. With expansion
it becomes easy to execute such things:
|exec prog1 {}$optlist1 | prog2 {}$optlist2 | prog3 -staticOption
As you can see, without expansion building this pipeline would be quite
a complex task and would make what is going on much more obscure.
With Tk, there are many examples. Here's one from the creation of lines
on a canvas:
|set id [$canv create polygon {}$coords -fill {} {}$opts -tags {foo bar}]
In this case, there is a fair amount of material before the coordinate
list, some static options (which act like defaults) between the coords
list and the user-supplied options list, and some further options (which
act like overrides for semantically-significant bits) after that which
need careful space handling. This also demonstrates why having a command
with a list of expanding indices is not a good idea, since it is plain to
see that ongoing maintenance might place extra non-expanding arguments in
various places through the command.
Another example demonstrates why having commands as well as variables
expanded is a good idea:
|namespace eval my {
| variable defaults {
| -bg white
| -fg black
| }| proc entry {path args} {
| variable defaults
| ::entry $path {}$defaults {}[platformOpts $::tcl_platform(platform)] {}$args
| }|}
This illustrates why just plain [[concat]] doesn't work, demonstrates
that having a way to do commands can be very useful, and also shows
that having multiple expansion operations in a row is potentially useful.
A final example (deleting discontiguous ranges of characters with a
particular tag) shows that this is not just useful in Tk for when
creating widgets and canvas items:
|$text delete {}[$text tag ranges $tagToCleanUp]
In this case, the obvious alternative:
|foreach {s e} [$text tag ranges $theTag] { $text delete $s $e }
is wrong, because the positions of the tags move with each deletion (by contrast, the [[$text delete]] operation is careful in this regard.)
~ Reference Implementation
Not done yet.
~ Copyright
This document has been placed in the public domain.

# TIP 144: Argument Expansion Syntax
Author: Peter Spjuth <peter.spjuth@space.se>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
Author: <dgp@users.sf.net>
State: Withdrawn
Type: Project
Vote: Pending
Created: 26-Jul-2003
Post-History:
Obsoleted-By: 157
Tcl-Version: 8.5
-----# Abstract
This TIP proposes to add syntax in Tcl to perform argument expansion
in a safe and efficient manner.
# Introduction
Many commands take a variable number of arguments and often you find
yourself with those arguments in a list. This list must then be
expanded into individual arguments to the command. This is currently
done with eval:
eval destroy [winfo children .]
This is a bit obscure and also very error prone when the command
becomes more complex. It is also inefficient and not object safe, why
something specialised in doing this would be better.
# Background
See also [[103]](103.md). Please also see a summary of a
poll of TCLCORE readers taken after [[103]](103.md) was
rejected.
<http://wiki.tcl.tk/9462># Rationale
For examples three statements are used. This is the eval version:
eval destroy [winfo children .]
eval button .b $stdargs -text \$mytext -bd $border
eval exec \$prog $opts1 [getMoreopts] \$file1 \$file2
The eval version would be even more complex if the lists that are to
be expanded are not known to be pure. To be really safe the last
would be:
eval exec \$prog [lrange $opts1 0 end] [lrange [getMoreopts] 0 end] \$file1 \$file2
With the proposed syntax they become:
destroy {}[winfo children .]
button .b {}$stdargs -text $mytext -bd $border
exec $prog {}$opts1 {}[getMoreopts] $file1 $file2
The advantage of using syntax for this is that the command do not get
obscured. In the examples destroy/button/exec is the most important
information on each line and it gets to be first on the line.
# Specification
If a word starts with a pair of braces, "\{\}", and is followed by a non
whitespace character it signifies argument expansion. The braces are
removed and the rest of the word is parsed and substituted as any other
word. The
character after the removed "\{\}" counts as a first character in the
rules about open braces and double quotes. After substitution, the
word is then parsed as
a list \(as if with _Tcl\_SplitList\(\)_ or _Tcl\_GetListFromObj_\) and
each element of the list is added to the command being built as a
separate word with no further parsing.
Before executing the command any word to be expanded is treated as a
list where each element becomes one separate argument to the command.
_Note 1:_ A word should really start with \{\} to trigger expansion
which means that words like these are not expanded:
cmd "{}$temp" \{}[something]
_Note 2:_ Expansion is typically most useful with words like:
cmd {}$var {}[somecmd $arg] {}$arr([cmd $arg])
But things like this are also legal:
cmd {}word {}$x,$y {}[foo]xy[apa] {}{apa bepa}
# Motivating Examples
Many of the examples that make this TIP the way it is come from either
advanced usage of commands like [exec] or from Tk.
Consider the case where you have lists of options for several external
commands that are to be executed together in a pipeline. With expansion
it becomes easy to execute such things:
exec prog1 {}$optlist1 | prog2 {}$optlist2 | prog3 -staticOption
As you can see, without expansion building this pipeline would be quite
a complex task and would make what is going on much more obscure.
With Tk, there are many examples. Here's one from the creation of lines
on a canvas:
set id [$canv create polygon {}$coords -fill {} {}$opts -tags {foo bar}]
In this case, there is a fair amount of material before the coordinate
list, some static options \(which act like defaults\) between the coords
list and the user-supplied options list, and some further options \(which
act like overrides for semantically-significant bits\) after that which
need careful space handling. This also demonstrates why having a command
with a list of expanding indices is not a good idea, since it is plain to
see that ongoing maintenance might place extra non-expanding arguments in
various places through the command.
Another example demonstrates why having commands as well as variables
expanded is a good idea:
namespace eval my {
variable defaults {
-bg white
-fg black
} proc entry {path args} {
variable defaults
::entry $path {}$defaults {}[platformOpts $::tcl_platform(platform)] {}$args
} }
This illustrates why just plain [concat] doesn't work, demonstrates
that having a way to do commands can be very useful, and also shows
that having multiple expansion operations in a row is potentially useful.
A final example \(deleting discontiguous ranges of characters with a
particular tag\) shows that this is not just useful in Tk for when
creating widgets and canvas items:
$text delete {}[$text tag ranges $tagToCleanUp]
In this case, the obvious alternative:
foreach {s e} [$text tag ranges $theTag] { $text delete $s $e }
is wrong, because the positions of the tags move with each deletion \(by contrast, the [$text delete] operation is careful in this regard.\)
# Reference Implementation
Not done yet.
# Copyright
This document has been placed in the public domain.

TIP: 145Title: Enhanced Tk Font Handling
Version: $Revision: 1.10 $
Author: Pat Thoyts <patthoyts@users.sourceforge.net>
State: Final
Type: Project
Vote: Done
Created: 31-Jul-2003
Post-History:
Obsoletes: 64
Tcl-Version: 8.5
~ Abstract
[64] suggests some improvements to font handling under windows.
However, not all of this TIP appears to have been implemented and I
believe this can be done better using the Tk named fonts mechanism.
~ Rationale
Windows, and no doubt MacOS as well, provide a number of system
defined fonts. These are fonts that may be configured by the end user
and that all applications are expected to use. The list includes the
menu font, the caption font, the tooltip font and others. Some of the
special fonts have been given symbolic names but these are not
available to dynamic inspection. This TIP proposes to make these
fonts available using the named font mechanism as used by the '''font
create''' command. This has some significant advantages. The set of
special font names becomes available to the programmer using the
'''font names''' command. The font configuration is available using the
'''font configure''' command. Also there is already a scheme in place
to update all widgets using a named font when that fonts configuration
changes. It is now a simple matter of re-defining the system fonts on
receipt of the WM_SETTINGCHANGE message to have Tk properly reflect
changes to the system-wide font selection.
~ Proposed Changes
~~ Generic changes
The named font creation routine '''CreateNamedFont()''' is to be exposed
on the public API as '''Tk_CreateNamedFont()'''. Also, the code used to
delete a named font should be factored out into an API function,
'''Tk_DeleteNamedFont()'''.
~~ Windows Platform-Specific Changes
A '''TkWinSetupSystemFonts()''' function is to be created which obtains
the font description for the system fonts and calls
'''Tk_CreateNamedFont()''' appropriately. This needs to be called during
font package initialization. The fonts are obtained using the Win32
''SystemParameters'' API and the set of defined stock object identifiers
used with ''GetStockObject''.
The toplevel Windows message message handler function '''WmProc''' must
handle the WM_SETTINGCHANGE message by calling the
'''TkWinSetupSystemFonts()''' function. This must call '''Tk_DeleteNamedFont'''
and then '''Tk_CreateNamedFont''' to properly re-define the font and to
propagate this change to all Tk widgets concerned. The Tk wigets will
properly handle the font update by calling their ''WorldChanged''
procedures.
~~ Cross-Platform Concerns
It seems likely to be useful to ensure that aliases are available for
each platform for the set of system fonts. This is currently done for
the Tk cursors and helps to ensure that a script written for one
system will usefully operate on another platform.
Tk currently guarantees that "''Courier''", "''Times''" and "''Helvetica''" will
always be available and are mapped to something suitable. It has been
proposed that these should really be "''monospace''", "''serif''" and
"''sans-serif''". The CSS specification
[http://www.w3.org/TR/REC-CSS2/fonts.html#generic-font-families]
defines these plus "''cursive''"
and "''fantasy''" but there seems no good reason to include these two
names.
~~ System Defined Fonts
~~~ Windows
The current set of special font names that Tk defines under windows
are as follows, all with -underline 0 -overstrike 0 -slant roman
| ansi -family {MS Sans Serif} -size 8 -weight normal
| ansifixed -family {Courier New} -size 10 -weight normal
| defaultgui -family {MS Shell Dlg} -size 8 -weight normal
| device -family System -size 10 -weight bold
| fixed -family {Courier New} -size 10 -weight normal
| oemfixed -family Terminal -size 9 -weight normal
| system -family System -size 10 -weight bold
| systemfixed -family Fixedsys -size 9 -weight normal
These are: ansi, windows variable pitch system font; ansifixed, a
monospace version of ansi; defaultgui, default GDI font for menus and
dialog boxes; system, used for system.....
We can additionally provide from Windows system settings
| TkCaptionFont -family {Trebuchet MS} -size 10 -weight bold
| TkIconFont -family Tahoma -size 8 -weight normal
| TkMenuFont -family Tahoma -size 8 -weight normal
| TkMessageFont -family Tahoma -size 8 -weight normal
| TkSmallCaptionFont -family Tahoma -size 8 -weight bold
| TkStatusFont -family Tahoma -size 8 -weight normal
These are fairly obvious. The caption font appears in the window title
bar, the icon font for desktop icons, message font for message boxes,
menu font, small caption font is for toolbox-type dialogs with very
thin title bars. Finally status font is the font used for tooltips and
status bars. All these Tk* fonts can be changed via the Windows
Desktop properties dialog.
~~~ Motif
| XmNbuttonFontList (for pushbuttons in dialogs)
| XmNlabelFontList (for labels)
| XmNtextFontList (for text entry fields)
| XmNdefaultFontList (for everything else)
~~~ GNOME
Gnome or GTK+ documentation, but the Gnome 2.2 "Font Preferences"
dialog lets you specify:
| Application font
| Desktop font
| Window title font
| Terminal font
It appears that all GTK+ UI controls use the same "Application" font.
~~~ KDE
KDE's font preferences dialog lists:
| General
| Fixed Width
| Toolbar
| Menu
| Window Title
| Taskbar
~~~ Aqua
Aqua defines the following
[http://developer.apple.com/documentation/UserExperience/Conceptual/AquaHIGuidelines/AHIGFonts/index.html]:
| System font
| Emphasized system font
| Small system font
| Emphasized small system font
| Application font
| Label font
| Mini system font
> The system font is used for text in menus, modeless dialogs, and
titles of document windows [...] The small system font is used for
informative text in alerts [...]. It is also the default font for
headings in lists, for help tags, and for text in the small
versions of many controls. If your application creates text
documents, use the application font as the default for user-created
content. The label font is used for labels with controls such as
sliders and icon bevel buttons. If necessary, the mini system font
can be used for utility window labels and text. Use emphasized
system fonts sparingly.
An alternative reference [http://developer.apple.com/documentation/Carbon/Reference/Appearance_Manager/appearance_manager/constant_15.html] lists:
| kThemeSystemFont
| kThemeSmallSystemFont
| kThemeSmallEmphasizedSystemFont
| kThemeViewsFont
| kThemeEmphasizedSystemFont
| kThemeApplicationFont
| kThemeLabelFont
| kThemeMenuTitleFont
| kThemeMenuItemFont
| kThemeMenuItemMarkFont
| kThemeMenuItemCmdKeyFont
| kThemeWindowTitleFont
| kThemePushButtonFont
| kThemeUtilityWindowTitleFont
| kThemeAlertHeaderFont
| kThemeSystemFontDetail
| kThemeSystemFontDetailEmphasized
| kThemeToolbarFont
~ Suggested Tk Font Aliases
From all the above this TIP suggests that the following names be provided
for all platforms and where possible these map to system defined fonts.
TkDefaultFont: the default for all GUI items not otherwise specified.
TkFixedFont: standard fixed width font
................................................................................
TkCaptionFont: used for window and dialog caption bars
TkSmallCaptionFont: used for captions on contained windows or tool
dialogs.
TkIconFont: font in use for icon captions
TkTooltipFont: a font to use for tooltip windows (transient
information windows)
~ Implementation
http://sf.net/tracker/?func=detail&aid=780617&group_id=12997&atid=312997~ Copyright
This document is placed in the public domain.

# TIP 145: Enhanced Tk Font Handling
Author: Pat Thoyts <patthoyts@users.sourceforge.net>
State: Final
Type: Project
Vote: Done
Created: 31-Jul-2003
Post-History:
Obsoletes: 64
Tcl-Version: 8.5
-----# Abstract
[[64]](64.md) suggests some improvements to font handling under windows.
However, not all of this TIP appears to have been implemented and I
believe this can be done better using the Tk named fonts mechanism.
# Rationale
Windows, and no doubt MacOS as well, provide a number of system
defined fonts. These are fonts that may be configured by the end user
and that all applications are expected to use. The list includes the
menu font, the caption font, the tooltip font and others. Some of the
special fonts have been given symbolic names but these are not
available to dynamic inspection. This TIP proposes to make these
fonts available using the named font mechanism as used by the **font
create** command. This has some significant advantages. The set of
special font names becomes available to the programmer using the
**font names** command. The font configuration is available using the
**font configure** command. Also there is already a scheme in place
to update all widgets using a named font when that fonts configuration
changes. It is now a simple matter of re-defining the system fonts on
receipt of the WM\_SETTINGCHANGE message to have Tk properly reflect
changes to the system-wide font selection.
# Proposed Changes
## Generic changes
The named font creation routine **CreateNamedFont\(\)** is to be exposed
on the public API as **Tk\_CreateNamedFont\(\)**. Also, the code used to
delete a named font should be factored out into an API function,
**Tk\_DeleteNamedFont\(\)**.
## Windows Platform-Specific Changes
A **TkWinSetupSystemFonts\(\)** function is to be created which obtains
the font description for the system fonts and calls
**Tk\_CreateNamedFont\(\)** appropriately. This needs to be called during
font package initialization. The fonts are obtained using the Win32
_SystemParameters_ API and the set of defined stock object identifiers
used with _GetStockObject_.
The toplevel Windows message message handler function **WmProc** must
handle the WM\_SETTINGCHANGE message by calling the
**TkWinSetupSystemFonts\(\)** function. This must call **Tk\_DeleteNamedFont**
and then **Tk\_CreateNamedFont** to properly re-define the font and to
propagate this change to all Tk widgets concerned. The Tk wigets will
properly handle the font update by calling their _WorldChanged_
procedures.
## Cross-Platform Concerns
It seems likely to be useful to ensure that aliases are available for
each platform for the set of system fonts. This is currently done for
the Tk cursors and helps to ensure that a script written for one
system will usefully operate on another platform.
Tk currently guarantees that "_Courier_", "_Times_" and "_Helvetica_" will
always be available and are mapped to something suitable. It has been
proposed that these should really be "_monospace_", "_serif_" and
"_sans-serif_". The CSS specification
<http://www.w3.org/TR/REC-CSS2/fonts.html#generic-font-families>
defines these plus "_cursive_"
and "_fantasy_" but there seems no good reason to include these two
names.
## System Defined Fonts
### Windows
The current set of special font names that Tk defines under windows
are as follows, all with -underline 0 -overstrike 0 -slant roman
ansi -family {MS Sans Serif} -size 8 -weight normal
ansifixed -family {Courier New} -size 10 -weight normal
defaultgui -family {MS Shell Dlg} -size 8 -weight normal
device -family System -size 10 -weight bold
fixed -family {Courier New} -size 10 -weight normal
oemfixed -family Terminal -size 9 -weight normal
system -family System -size 10 -weight bold
systemfixed -family Fixedsys -size 9 -weight normal
These are: ansi, windows variable pitch system font; ansifixed, a
monospace version of ansi; defaultgui, default GDI font for menus and
dialog boxes; system, used for system.....
We can additionally provide from Windows system settings
TkCaptionFont -family {Trebuchet MS} -size 10 -weight bold
TkIconFont -family Tahoma -size 8 -weight normal
TkMenuFont -family Tahoma -size 8 -weight normal
TkMessageFont -family Tahoma -size 8 -weight normal
TkSmallCaptionFont -family Tahoma -size 8 -weight bold
TkStatusFont -family Tahoma -size 8 -weight normal
These are fairly obvious. The caption font appears in the window title
bar, the icon font for desktop icons, message font for message boxes,
menu font, small caption font is for toolbox-type dialogs with very
thin title bars. Finally status font is the font used for tooltips and
status bars. All these Tk\* fonts can be changed via the Windows
Desktop properties dialog.
### Motif
XmNbuttonFontList (for pushbuttons in dialogs)
XmNlabelFontList (for labels)
XmNtextFontList (for text entry fields)
XmNdefaultFontList (for everything else)
### GNOME
Gnome or GTK\+ documentation, but the Gnome 2.2 "Font Preferences"
dialog lets you specify:
Application font
Desktop font
Window title font
Terminal font
It appears that all GTK\+ UI controls use the same "Application" font.
### KDE
KDE's font preferences dialog lists:
General
Fixed Width
Toolbar
Menu
Window Title
Taskbar
### Aqua
Aqua defines the following
<http://developer.apple.com/documentation/UserExperience/Conceptual/AquaHIGuidelines/AHIGFonts/index.html> :
System font
Emphasized system font
Small system font
Emphasized small system font
Application font
Label font
Mini system font
> The system font is used for text in menus, modeless dialogs, and
titles of document windows [...] The small system font is used for
informative text in alerts [...]. It is also the default font for
headings in lists, for help tags, and for text in the small
versions of many controls. If your application creates text
documents, use the application font as the default for user-created
content. The label font is used for labels with controls such as
sliders and icon bevel buttons. If necessary, the mini system font
can be used for utility window labels and text. Use emphasized
system fonts sparingly.
An alternative reference <http://developer.apple.com/documentation/Carbon/Reference/Appearance_Manager/appearance_manager/constant_15.html> lists:
kThemeSystemFont
kThemeSmallSystemFont
kThemeSmallEmphasizedSystemFont
kThemeViewsFont
kThemeEmphasizedSystemFont
kThemeApplicationFont
kThemeLabelFont
kThemeMenuTitleFont
kThemeMenuItemFont
kThemeMenuItemMarkFont
kThemeMenuItemCmdKeyFont
kThemeWindowTitleFont
kThemePushButtonFont
kThemeUtilityWindowTitleFont
kThemeAlertHeaderFont
kThemeSystemFontDetail
kThemeSystemFontDetailEmphasized
kThemeToolbarFont
# Suggested Tk Font Aliases
From all the above this TIP suggests that the following names be provided
for all platforms and where possible these map to system defined fonts.
TkDefaultFont: the default for all GUI items not otherwise specified.
TkFixedFont: standard fixed width font
................................................................................
TkCaptionFont: used for window and dialog caption bars
TkSmallCaptionFont: used for captions on contained windows or tool
dialogs.
TkIconFont: font in use for icon captions
TkTooltipFont: a font to use for tooltip windows \(transient
information windows\)
# Implementation
<http://sf.net/tracker/?func=detail&aid=780617&group\_id=12997&atid=312997># Copyright
This document is placed in the public domain.

TIP: 146Title: Add Overall Anchoring to the Grid Geometry Manager
Version: $Revision: 1.16 $
Author: Peter Spjuth <peter.spjuth@space.se>
State: Final
Type: Project
Vote: Done
Created: 05-Aug-2003
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes to add an anchor option to grid managers to control
the behaviour of a grid where all weights are zero.
~ Rationale
Have you ever found yourself adding "-weight" to an unused row or
column in a grid? That is usually the workaround to use when you have
no "-weight" on any column but you don't want things to sit in the
middle of the grid when the window is grown.
By adding an anchor option it lets the code directly reflect the
intention of the programmer and it makes the code more maintainable
since you can add and remove columns without having to update the
dummy weight.
~ Shrinking a Grid
When growing a grid, the behaviour is rather simple. Follow the
weights and when 0, follow the anchor.
When shrinking a grid without weight things get more interesting.
The previous behaviour of the grid is a bit inconsistent in that when
grown it centers the slaves (as in anchor "center") and when shrunk it
shows the upper left part (as in anchor "nw").
Thus, following the new anchor will not be backwardly compatible, even
though I doubt it can be common for people to rely on that behaviour.
For shrinking a grid without weight there are three options.
1. Always clip the bottom/right. i.e. behave as now.
2. Clip according to anchor.
3. Try to shrink all columns/rows, and only clip when minsize stops
further shrinking. (Clip according to anchor)
I think 2. makes more sense than 1., but it does mean a minor
compatiblity breach. Also, with 2., behaviour 1. and 3. can be emulated
but the other way around is harder.
With 2, there are also two choises how to break compatibility depending
on the default anchor.
2. a With default anchor ''center''. Anyone relying on the
bottom/left clipping will find things clip all around. Potentially an
important widget in the nw corner can end up outside.
2. b With default anchor ''nw''. Anyone relying on the centering will
find things ending up in the nw corner. Probably just a visual
difference.
I can't see that any of those two are significantly worse than the other.
Finally, the question "what would you do if backwards compatiblity was
not an aspect?" clearly answers "2b",
leading to the specification in this TIP.
~ Specification
A new subcommand ''grid anchor'' is added. It is similar to ''grid
propagate'' in that it configures an aspect of a grid manager.
The syntax is ''grid anchor master ?value?'' where ''value'' is a
standard anchor value with ''nw'' as default.
Whenever there is extra space in the grid manager and all weights
are zero, the layout is placed within its master according to the
master's anchor value.
Whenever there is too little space in the grid manager and all weights
are zero or all columns have reached their minimumn size, the layout is
clipped according to the master's anchor value.
~ Shrink example
To clarify the shrinking options here is an example.
a: Current behaviour or, "1" with anchor "center".
b: "2" with anchor "center"
c: "2" with anchor "nw"
d: "3" with anchor "nw"
|foreach top {.a .b .c .d} {
| toplevel $top
| frame $top.f
| for {set row 0} {$row < 2} {incr row} {
| for {set col 0} {$col < 4} {incr col} {
| set w $top.f.b${row}x$col
| button $w -text HejHopp
| grid $w -row $row -column $col
| } | } |} ||# Current behaviour
|pack .a.f -fill both -expand 1
|# Anchor "center" with clipping
|place .b.f -relx 0.5 -rely 0.5 -anchor center
|wm geometry .b 150x50
|# Anchor "nw" with clipping
|pack .c.f -side top -anchor w
|# Anchor "nw" with shrinking
|pack .d.f -side top -anchor w
|grid rowconfigure .d.f {0 1} -weight 1
|grid columnconfigure .d.f {0 1 2 3} -weight 1
~ Reference Implementation
Not implemented yet.
~ Copyright
This document has been placed in the public domain.

# TIP 146: Add Overall Anchoring to the Grid Geometry Manager
Author: Peter Spjuth <peter.spjuth@space.se>
State: Final
Type: Project
Vote: Done
Created: 05-Aug-2003
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes to add an anchor option to grid managers to control
the behaviour of a grid where all weights are zero.
# Rationale
Have you ever found yourself adding "-weight" to an unused row or
column in a grid? That is usually the workaround to use when you have
no "-weight" on any column but you don't want things to sit in the
middle of the grid when the window is grown.
By adding an anchor option it lets the code directly reflect the
intention of the programmer and it makes the code more maintainable
since you can add and remove columns without having to update the
dummy weight.
# Shrinking a Grid
When growing a grid, the behaviour is rather simple. Follow the
weights and when 0, follow the anchor.
When shrinking a grid without weight things get more interesting.
The previous behaviour of the grid is a bit inconsistent in that when
grown it centers the slaves \(as in anchor "center"\) and when shrunk it
shows the upper left part \(as in anchor "nw"\).
Thus, following the new anchor will not be backwardly compatible, even
though I doubt it can be common for people to rely on that behaviour.
For shrinking a grid without weight there are three options.
1. Always clip the bottom/right. i.e. behave as now.
2. Clip according to anchor.
3. Try to shrink all columns/rows, and only clip when minsize stops
further shrinking. \(Clip according to anchor\)
I think 2. makes more sense than 1., but it does mean a minor
compatiblity breach. Also, with 2., behaviour 1. and 3. can be emulated
but the other way around is harder.
With 2, there are also two choises how to break compatibility depending
on the default anchor.
2. a With default anchor _center_. Anyone relying on the
bottom/left clipping will find things clip all around. Potentially an
important widget in the nw corner can end up outside.
2. b With default anchor _nw_. Anyone relying on the centering will
find things ending up in the nw corner. Probably just a visual
difference.
I can't see that any of those two are significantly worse than the other.
Finally, the question "what would you do if backwards compatiblity was
not an aspect?" clearly answers "2b",
leading to the specification in this TIP.
# Specification
A new subcommand _grid anchor_ is added. It is similar to _grid
propagate_ in that it configures an aspect of a grid manager.
The syntax is _grid anchor master ?value?_ where _value_ is a
standard anchor value with _nw_ as default.
Whenever there is extra space in the grid manager and all weights
are zero, the layout is placed within its master according to the
master's anchor value.
Whenever there is too little space in the grid manager and all weights
are zero or all columns have reached their minimumn size, the layout is
clipped according to the master's anchor value.
# Shrink example
To clarify the shrinking options here is an example.
a: Current behaviour or, "1" with anchor "center".
b: "2" with anchor "center"
c: "2" with anchor "nw"
d: "3" with anchor "nw"
foreach top {.a .b .c .d} {
toplevel $top
frame $top.f
for {set row 0} {$row < 2} {incr row} {
for {set col 0} {$col < 4} {incr col} {
set w $top.f.b${row}x$col
button $w -text HejHopp
grid $w -row $row -column $col
} } } # Current behaviour
pack .a.f -fill both -expand 1
# Anchor "center" with clipping
place .b.f -relx 0.5 -rely 0.5 -anchor center
wm geometry .b 150x50
# Anchor "nw" with clipping
pack .c.f -side top -anchor w
# Anchor "nw" with shrinking
pack .d.f -side top -anchor w
grid rowconfigure .d.f {0 1} -weight 1
grid columnconfigure .d.f {0 1 2 3} -weight 1
# Reference Implementation
Not implemented yet.
# Copyright
This document has been placed in the public domain.

TIP: 147Title: Make Grid's Column/Row Configure Easier
Version: $Revision: 1.5 $
Author: Peter Spjuth <peter.spjuth@space.se>
State: Final
Type: Project
Vote: Done
Created: 05-Aug-2003
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes to add an alternative way to state which
columns/rows in a grid are affected by an column/rowconfigure command.
~ Rationale
Usually when doing a grid layout I tend to think in terms of what I
want to do with a particular widget. For example a canvas or a frame
that I want to expand within a resized window. Currently I need to
translate that intention into a column and a row number to pass to
column/rowconfigure. A more direct approach would be to pass the
widget name directly to column/rowconfigure.
Another situation is when all columns should be affected. Here I
currently need to list them all {0 1 2 3 4 5}. Allowing an "all"
keyword makes it easier and the code becomes more maintainable since
you can add/remove columns without updating the list.
What should be done when columnspan is greater than 1?
There are three obvious possibilities: affect all, affect the first
and affect the last. Any of these may be wanted by a programmer at
one time or another so whatever is chosen, someone will be annoyed
with it. Affecting all seems most logical.
See also
http://sf.net/tracker/?func=detail&aid=659218&group_id=12997&atid=362997~ Specification
Anywhere ''column'' is used below, the same applies to ''row'' too.
In the ''index'' argument to ''grid columnconfigure'', widget names
and the keyword "all" is recognised besides the normal integers.
When a widget name is used, it must be currently managed by the given
''master''. The columns currently occupied by the widget are affected
by the command.
When the keyword "all" is used, all columns currently occupied by
widgets in ''master'' are affected by the command.
''Note:'' "Currently occupied" means at the time of the invocation of
the ''grid columnconfigure'' command. The grid does not try to trace
if a widget is moved in the grid or if more widgets are added.
~ Examples
|canvas .c
|scrollbar .sbx -orient horizontal
|scrollbar .sby -orient vertical
|grid .c .sby -sticky news
|grid .sbx -sticky we
|grid columnconfigure . .c -weight 1
|grid rowconfigure . .c -weight 1
||button .b1 -text a
|button .b2 -text b
|button .b3 -text example
|button .b4 -text xyzzy
|grid .b1 .b2 .b3 .b4 -sticky news
|grid columnconfigure . all -uniform a -weight 1
~ Other Possibilities
One possibility that is not proposed in this TIP is to also recognise
the end-style indexing that indices in Tcl support (through
''TclGetIntForIndex()''.)
Comment: "Personally, I think that end-index handling will be quite
tricky since it would require knowing what the *user* thinks is the
last column."
~ Reference Implementation
Not implemented yet.
~ Copyright
This document has been placed in the public domain.

# TIP 147: Make Grid's Column/Row Configure Easier
Author: Peter Spjuth <peter.spjuth@space.se>
State: Final
Type: Project
Vote: Done
Created: 05-Aug-2003
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes to add an alternative way to state which
columns/rows in a grid are affected by an column/rowconfigure command.
# Rationale
Usually when doing a grid layout I tend to think in terms of what I
want to do with a particular widget. For example a canvas or a frame
that I want to expand within a resized window. Currently I need to
translate that intention into a column and a row number to pass to
column/rowconfigure. A more direct approach would be to pass the
widget name directly to column/rowconfigure.
Another situation is when all columns should be affected. Here I
currently need to list them all \{0 1 2 3 4 5\}. Allowing an "all"
keyword makes it easier and the code becomes more maintainable since
you can add/remove columns without updating the list.
What should be done when columnspan is greater than 1?
There are three obvious possibilities: affect all, affect the first
and affect the last. Any of these may be wanted by a programmer at
one time or another so whatever is chosen, someone will be annoyed
with it. Affecting all seems most logical.
See also
<http://sf.net/tracker/?func=detail&aid=659218&group\_id=12997&atid=362997># Specification
Anywhere _column_ is used below, the same applies to _row_ too.
In the _index_ argument to _grid columnconfigure_, widget names
and the keyword "all" is recognised besides the normal integers.
When a widget name is used, it must be currently managed by the given
_master_. The columns currently occupied by the widget are affected
by the command.
When the keyword "all" is used, all columns currently occupied by
widgets in _master_ are affected by the command.
_Note:_ "Currently occupied" means at the time of the invocation of
the _grid columnconfigure_ command. The grid does not try to trace
if a widget is moved in the grid or if more widgets are added.
# Examples
canvas .c
scrollbar .sbx -orient horizontal
scrollbar .sby -orient vertical
grid .c .sby -sticky news
grid .sbx -sticky we
grid columnconfigure . .c -weight 1
grid rowconfigure . .c -weight 1
button .b1 -text a
button .b2 -text b
button .b3 -text example
button .b4 -text xyzzy
grid .b1 .b2 .b3 .b4 -sticky news
grid columnconfigure . all -uniform a -weight 1
# Other Possibilities
One possibility that is not proposed in this TIP is to also recognise
the end-style indexing that indices in Tcl support \(through
_TclGetIntForIndex\(\)_.\)
Comment: "Personally, I think that end-index handling will be quite
tricky since it would require knowing what the \*user\* thinks is the
last column."
# Reference Implementation
Not implemented yet.
# Copyright
This document has been placed in the public domain.

TIP: 148Title: Correct [list]-Quoting of the '#' Character
Version: $Revision: 1.5 $
Author: Don Porter <dgp@users.sf.net>
State: Final
Type: Project
Vote: Done
Created: 08-Aug-2003
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes the correction of a long-standing bug in the
[[list]]-quoting of the ''#'' character.
~ Background
Tcl has a bug in its [[list]]-quoting function. The bug is recorded
as Tcl Bug 489537
(http://sf.net/tracker/?func=detail&aid=489537&group_id=10894&atid=110894).
Briefly, one of the documented functions of [[list]] is quoting of
arguments so the result can be passed to [[eval]] with each list
element becoming exactly one word of the command to be evaluated.
Consider the example script:
| # FILE: demo.tcl
| set cmdName [lindex $argv 0]
| proc $cmdName {} {puts Success!}
| set command [list $cmdName]
| puts "Evaluating \[$command]..."
| eval $command
This script expects one argument on the command line, and should write
''Success!'' to stdout. This script works correctly for most input:
| $ tclsh demo.tcl foo
| Evaluating [foo]...
| Success!
| $ tclsh demo.tcl "with space"
| Evaluating [{with space}]...
| Success!
But it fails for any argument beginning with the ''#'' character:
| $ tclsh demo.tcl #bar
| Evaluating [#bar]...
This is because, contrary to the documentation, [[list]] does not
quote the leading ''#'' in a manner to make the list safe for passing
to [[eval]]. The Tcl parser sees the unquoted ''#'' as the start of a
comment.
Starting in Tcl 8.3, optimizations for evaluation of ''pure lists''
were added, making inconsistency due to ''Tcl_ObjType'' shimmering a
new symptom of this bug. If we adapt the example script to remove the
[[puts]] (so that a ''pure list'' is maintained):
| # FILE: demo2.tcl
| set cmdName [lindex $argv 0]
| proc $cmdName {} {puts Success!}
| set command [list $cmdName]
| eval $command
We get a script that actually works with the troublesome input:
| $ tclsh demo2.tcl #bar
| Success!
This bug in [[list]]-quoting is present in all released versions of
Tcl since and including Tcl 7.4. It may go back further.
There is no question that Tcl's behavior disagrees with its
documentation on this point. I believe the documentation to be
correct. From that viewpoint, this is a bug, not requiring a TIP for
fixing. Because the bug has been around for so long, though, it seems
prudent to make the TIP proposal, if only as fair warning to those who
might have bug-dependent scripts to fix. The particular fix proposed
also adds a single ''#define'' to Tcl's public header file.
A large number of tests have been added to the Tcl test suite in the
HEAD, demonstrating this bug in several ways.
~ Proposal
''Tcl_ConvertCountedElement()'' is modified to have the default
behavior of quoting any leading ''#'' character in a list element.
With this default quoting, any string representation of a list
generated by Tcl will not begin with the ''#'' character, so cannot be
mis-parsed as a comment.
''Tcl_ConvertCountedElement()'' is also modified to recognize a new
bit flag value in its ''flags'' argument, ''TCL_DONT_QUOTE_HASH'',
which is defined in Tcl's public header file so that it may be used by
extensions. When the ''TCL_DONT_QUOTE_HASH'' bit is set in the
''flags'' argument, ''Tcl_ConvertCountedElement()'' will not quote the
leading ''#'' character. Quoting of the leading ''#'' character is
only necessary for the first element of a list. Those callers of
''Tcl_ConvertCountedElement()'' that can be sure they are not
generating the first element of a list can pass in the
''TCL_DONT_QUOTE_HASH'' bit to produce the simplest quoting required.
The behavior of the ''TCL_DONT_QUOTE_HASH'' bit flag is added to the
documentation. The ''Tcl_ConvertElement()'' routine is similarly
modified (trivially, since it is just a wrapper).
All callers of ''Tcl_ConvertCountedElement()'' in the Tcl source code
are modified to use the ''TCL_DONT_QUOTE_HASH'' flag as appropriate,
so that Tcl continues to generate as simple string representations of
lists as possible that do not suffer from Bug 489537.
~ Prototype
A patch implementing this proposal is attached to Tcl Bug 489537 at
SourceForge.
~ Compatibility
After acceptance of this patch, the string representation of some
lists will change, though as little as possible while still fixing the
bug. Scripts that perform string comparisons on lists may see
different results. Notably, a test in a test suite that has a test
body that generates a list, and then has an expected result as a
string may see new test failures. The minimal quoting changes should
keep this incompatibility to a minimum, but it may happen.
Notably there are no such compatibility problems in either the Tcl or
Tk test suites. Any such incompatibility in other test suites can be
easily remedied by using [[list]] to generate the expected result.
~ Scope
It has been observed by some Tcl users that [[list]] is used for two
conceptually distinct purposes. First, adding quoting to list
elements as required, so that element boundaries can be re-discovered
from the string representation. Second, adding quoting so that the
string representation can be passed to [[eval]] with the original
element boundaries becoming the argument boundaries in the evaluation.
One can imagine a Tcl where these two functions were separated.
However, this TIP does not propose such a separation, and further
arguments on that point are out of scope, and should be considered in
another TIP, if at all.
~ Acknowledgements
The author acknowledges the discovery of this bug by Joe English,
analysis by Donal Fellows, and a first draft patch from Miguel Sofer.
~ Copyright
This document has been placed in the public domain.

# TIP 148: Correct [list]-Quoting of the '#' Character
Author: Don Porter <dgp@users.sf.net>
State: Final
Type: Project
Vote: Done
Created: 08-Aug-2003
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes the correction of a long-standing bug in the
[list]-quoting of the _\#_ character.
# Background
Tcl has a bug in its [list]-quoting function. The bug is recorded
as Tcl Bug 489537
\(<http://sf.net/tracker/?func=detail&aid=489537&group\_id=10894&atid=110894\).>
Briefly, one of the documented functions of [list] is quoting of
arguments so the result can be passed to [eval] with each list
element becoming exactly one word of the command to be evaluated.
Consider the example script:
# FILE: demo.tcl
set cmdName [lindex $argv 0]
proc $cmdName {} {puts Success!}
set command [list $cmdName]
puts "Evaluating \[$command]..."
eval $command
This script expects one argument on the command line, and should write
_Success!_ to stdout. This script works correctly for most input:
$ tclsh demo.tcl foo
Evaluating [foo]...
Success!
$ tclsh demo.tcl "with space"
Evaluating [{with space}]...
Success!
But it fails for any argument beginning with the _\#_ character:
$ tclsh demo.tcl #bar
Evaluating [#bar]...
This is because, contrary to the documentation, [list] does not
quote the leading _\#_ in a manner to make the list safe for passing
to [eval]. The Tcl parser sees the unquoted _\#_ as the start of a
comment.
Starting in Tcl 8.3, optimizations for evaluation of _pure lists_
were added, making inconsistency due to _Tcl\_ObjType_ shimmering a
new symptom of this bug. If we adapt the example script to remove the
[puts] \(so that a _pure list_ is maintained\):
# FILE: demo2.tcl
set cmdName [lindex $argv 0]
proc $cmdName {} {puts Success!}
set command [list $cmdName]
eval $command
We get a script that actually works with the troublesome input:
$ tclsh demo2.tcl #bar
Success!
This bug in [list]-quoting is present in all released versions of
Tcl since and including Tcl 7.4. It may go back further.
There is no question that Tcl's behavior disagrees with its
documentation on this point. I believe the documentation to be
correct. From that viewpoint, this is a bug, not requiring a TIP for
fixing. Because the bug has been around for so long, though, it seems
prudent to make the TIP proposal, if only as fair warning to those who
might have bug-dependent scripts to fix. The particular fix proposed
also adds a single _\#define_ to Tcl's public header file.
A large number of tests have been added to the Tcl test suite in the
HEAD, demonstrating this bug in several ways.
# Proposal
_Tcl\_ConvertCountedElement\(\)_ is modified to have the default
behavior of quoting any leading _\#_ character in a list element.
With this default quoting, any string representation of a list
generated by Tcl will not begin with the _\#_ character, so cannot be
mis-parsed as a comment.
_Tcl\_ConvertCountedElement\(\)_ is also modified to recognize a new
bit flag value in its _flags_ argument, _TCL\_DONT\_QUOTE\_HASH_,
which is defined in Tcl's public header file so that it may be used by
extensions. When the _TCL\_DONT\_QUOTE\_HASH_ bit is set in the
_flags_ argument, _Tcl\_ConvertCountedElement\(\)_ will not quote the
leading _\#_ character. Quoting of the leading _\#_ character is
only necessary for the first element of a list. Those callers of
_Tcl\_ConvertCountedElement\(\)_ that can be sure they are not
generating the first element of a list can pass in the
_TCL\_DONT\_QUOTE\_HASH_ bit to produce the simplest quoting required.
The behavior of the _TCL\_DONT\_QUOTE\_HASH_ bit flag is added to the
documentation. The _Tcl\_ConvertElement\(\)_ routine is similarly
modified \(trivially, since it is just a wrapper\).
All callers of _Tcl\_ConvertCountedElement\(\)_ in the Tcl source code
are modified to use the _TCL\_DONT\_QUOTE\_HASH_ flag as appropriate,
so that Tcl continues to generate as simple string representations of
lists as possible that do not suffer from Bug 489537.
# Prototype
A patch implementing this proposal is attached to Tcl Bug 489537 at
SourceForge.
# Compatibility
After acceptance of this patch, the string representation of some
lists will change, though as little as possible while still fixing the
bug. Scripts that perform string comparisons on lists may see
different results. Notably, a test in a test suite that has a test
body that generates a list, and then has an expected result as a
string may see new test failures. The minimal quoting changes should
keep this incompatibility to a minimum, but it may happen.
Notably there are no such compatibility problems in either the Tcl or
Tk test suites. Any such incompatibility in other test suites can be
easily remedied by using [list] to generate the expected result.
# Scope
It has been observed by some Tcl users that [list] is used for two
conceptually distinct purposes. First, adding quoting to list
elements as required, so that element boundaries can be re-discovered
from the string representation. Second, adding quoting so that the
string representation can be passed to [eval] with the original
element boundaries becoming the argument boundaries in the evaluation.
One can imagine a Tcl where these two functions were separated.
However, this TIP does not propose such a separation, and further
arguments on that point are out of scope, and should be considered in
another TIP, if at all.
# Acknowledgements
The author acknowledges the discovery of this bug by Joe English,
analysis by Donal Fellows, and a first draft patch from Miguel Sofer.
# Copyright
This document has been placed in the public domain.

TIP: 149Title: Allow "enabled" as Synonym for "normal" in -state Option
Author: Michael A. Cleverly <michael@cleverly.com>
State: Withdrawn
Type: Project
Vote: Pending
Created: 13-Aug-2003
Tcl-Version: 8.5
Version: $Revision: 1.2 $
Post-History:
~ Abstract
This TIP allows Tk widgets which have a configurable ''-state'' option
to accept ''enabled'' as a synonym for ''normal''.~ Rationale
Tk widgets which have a configurable ''-state'' option have (at a
minimum) ''normal'' and ''disabled'' states. In common usage,
however, the antonym of ''disabled'' is ''enabled'', not ''normal''.
Even experienced Tcl/Tk programmers sometimes forget, though they
usually won't come out and admit it explicitly. (Cf.
http://groups.google.com/groups?th=66e286d7efc3bbb6)~ Proposed Changes
Modify all Tk widgets which have a configurable ''-state'' option to
accept ''enabled'' as a synonym for ''normal''. The value returned by
a widget's ''cget'' method would continue to be ''normal''. The
appropriate man pages and test suite will be updated to document and
test the use of ''enabled'' as a synonym.
|% $w configure -state enabled
|% puts [$w cget -state]
|normal
~ Implementation
A reference implementation will be provided shortly.
~ Copyright
This document is placed in the public domain.

# TIP 149: Allow "enabled" as Synonym for "normal" in -state Option
Author: Michael A. Cleverly <michael@cleverly.com>
State: Withdrawn
Type: Project
Vote: Pending
Created: 13-Aug-2003
Tcl-Version: 8.5
Post-History:
-----# Abstract
This TIP allows Tk widgets which have a configurable _-state_ option
to accept _enabled_ as a synonym for _normal_.# Rationale
Tk widgets which have a configurable _-state_ option have \(at a
minimum\) _normal_ and _disabled_ states. In common usage,
however, the antonym of _disabled_ is _enabled_, not _normal_.
Even experienced Tcl/Tk programmers sometimes forget, though they
usually won't come out and admit it explicitly. \(Cf.
<http://groups.google.com/groups?th=66e286d7efc3bbb6\)># Proposed Changes
Modify all Tk widgets which have a configurable _-state_ option to
accept _enabled_ as a synonym for _normal_. The value returned by
a widget's _cget_ method would continue to be _normal_. The
appropriate man pages and test suite will be updated to document and
test the use of _enabled_ as a synonym.
% $w configure -state enabled
% puts [$w cget -state]
normal
# Implementation
A reference implementation will be provided shortly.
# Copyright
This document is placed in the public domain.

TIP: 15Title: Functions to List and Detail Math Functions
Version: $Revision: 1.8 $
Author: Donal K. Fellows <fellowsd@cs.man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 22-Nov-2000
Post-History:
Keywords: Tcl,expr,function,introspection
Tcl-Version: 8.4.0
~ Abstract
Provides a way for the list of all math functions defined in the
current interpreter to be discovered, and for discovering what
arguments might be passed to an existing math function. This may be
useful in tests as well as more general use.
~ Rationale
Although it is quite easy to define a new function for use in
expressions, there is no public way of performing introspection on
this information. Having a way to extract the arguments from an
existing math function was requested by
http://sourceforge.net/bugs/?func=detailbug&bug_id=119304&group_id=10894
and once you have one, it becomes trivial to also ask for a second
function to list what functions are defined.
I propose the creation of two functions that fulfil this rôle;
''Tcl_GetMathFuncInfo'' and ''Tcl_ListMathFuncs''. These functions
will be documented on the same manual page as ''Tcl_CreateMathFunc''
and implemented in the same file.
Furthermore, I also propose that the ''info'' command in the Tcl
interpreter be extended to include a new subcommand, ''functions'',
which will allow Tcl scripts to discover the list of installed
functions (by acting as a thin veneer over ''Tcl_ListMathFuncs''.)
Note that this is an extension of the ''info'' command because it
allows for introspection of a system that affects the behaviour of
several commands that form the core part of the command-set: ''expr'',
''for'', ''if'' and ''while''.
~ Tcl_GetMathFuncInfo
This function will take an interpreter reference, a function name (as
a string) and pointers to variables capable of taking each of the last
four arguments to ''Tcl_CreateMathFunc'', and will return a standard
Tcl result (either ''TCL_OK'' or ''TCL_ERROR'', depending on whether a
function with the given name exists within the given interpreter, with
an error message being left in the interpreter's result in the
''TCL_ERROR'' case.) The array of argument types whose reference is
placed into the variable pointed to by ''argTypesPtr'' will be
allocated by Tcl, and should be freed with ''Tcl_Free''.
|int Tcl_GetMathFuncInfo(Tcl_Interp *interp, CONST char *name,
| int *numArgsPtr, Tcl_ValueType **argTypesPtr,
| Tcl_MathProc **procPtr,
| ClientData *clientDataPtr);
The parameter names are chosen by analogy with ''Tcl_CreateMathFunc''.
In the case where a math function is defined internally by the bytecode
engine and has no standard implementation (all the builtin functions in
8.4a2 are like this) the value placed in the variable indicated by the
''procPtr'' argument will be NULL.
~ Tcl_ListMathFuncs
This function will take an interpreter reference and an optional
string that describes a glob-like pattern that restricts the set of
math functions that the caller is interested in receiving (with a
''NULL'' indicating that no filtering is desired.) The function will
return a pointer to a newly-allocated ''Tcl_Obj'' list of the names of
all the math functions defined within that interpreter, or ''NULL'' in
the case of an error (in which case a suitable message will be left in
the interpreter.) The list will not be required to be sorted.
|Tcl_Obj *Tcl_ListMathFuncs(Tcl_Interp *interp, CONST char *pattern);
The alternative is to pass in the addresses of variables that will be
updated to contain the number of functions and an array of function
names. But I prefer the ''Tcl_Obj'' approach as it is expressing the
fact that the list of function names is really a single thing being
returned (albeit one that is not a simple value.) It is not
anticipated that the performance of this function will need to be
crucial to too many applications.
~ info functions
This new subcommand will provide access from Tcl scripts to the
functionality of ''Tcl_ListMathFuncs''. It will take a single optional
argument consisting of a pattern to pass on as the ''pattern'' argument
(with the absence of the argument indicating that NULL is to be passed
instead.)
~ Copyright
This document is placed in the public domain.

# TIP 15: Functions to List and Detail Math Functions
Author: Donal K. Fellows <fellowsd@cs.man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 22-Nov-2000
Post-History:
Keywords: Tcl,expr,function,introspection
Tcl-Version: 8.4.0
-----# Abstract
Provides a way for the list of all math functions defined in the
current interpreter to be discovered, and for discovering what
arguments might be passed to an existing math function. This may be
useful in tests as well as more general use.
# Rationale
Although it is quite easy to define a new function for use in
expressions, there is no public way of performing introspection on
this information. Having a way to extract the arguments from an
existing math function was requested by
<http://sourceforge.net/bugs/?func=detailbug&bug\_id=119304&group\_id=10894>
and once you have one, it becomes trivial to also ask for a second
function to list what functions are defined.
I propose the creation of two functions that fulfil this rÃ´le;
_Tcl\_GetMathFuncInfo_ and _Tcl\_ListMathFuncs_. These functions
will be documented on the same manual page as _Tcl\_CreateMathFunc_
and implemented in the same file.
Furthermore, I also propose that the _info_ command in the Tcl
interpreter be extended to include a new subcommand, _functions_,
which will allow Tcl scripts to discover the list of installed
functions \(by acting as a thin veneer over _Tcl\_ListMathFuncs_.\)
Note that this is an extension of the _info_ command because it
allows for introspection of a system that affects the behaviour of
several commands that form the core part of the command-set: _expr_,
_for_, _if_ and _while_.
# Tcl\_GetMathFuncInfo
This function will take an interpreter reference, a function name \(as
a string\) and pointers to variables capable of taking each of the last
four arguments to _Tcl\_CreateMathFunc_, and will return a standard
Tcl result \(either _TCL\_OK_ or _TCL\_ERROR_, depending on whether a
function with the given name exists within the given interpreter, with
an error message being left in the interpreter's result in the
_TCL\_ERROR_ case.\) The array of argument types whose reference is
placed into the variable pointed to by _argTypesPtr_ will be
allocated by Tcl, and should be freed with _Tcl\_Free_.
int Tcl_GetMathFuncInfo(Tcl_Interp *interp, CONST char *name,
int *numArgsPtr, Tcl_ValueType **argTypesPtr,
Tcl_MathProc **procPtr,
ClientData *clientDataPtr);
The parameter names are chosen by analogy with _Tcl\_CreateMathFunc_.
In the case where a math function is defined internally by the bytecode
engine and has no standard implementation \(all the builtin functions in
8.4a2 are like this\) the value placed in the variable indicated by the
_procPtr_ argument will be NULL.
# Tcl\_ListMathFuncs
This function will take an interpreter reference and an optional
string that describes a glob-like pattern that restricts the set of
math functions that the caller is interested in receiving \(with a
_NULL_ indicating that no filtering is desired.\) The function will
return a pointer to a newly-allocated _Tcl\_Obj_ list of the names of
all the math functions defined within that interpreter, or _NULL_ in
the case of an error \(in which case a suitable message will be left in
the interpreter.\) The list will not be required to be sorted.
Tcl_Obj *Tcl_ListMathFuncs(Tcl_Interp *interp, CONST char *pattern);
The alternative is to pass in the addresses of variables that will be
updated to contain the number of functions and an array of function
names. But I prefer the _Tcl\_Obj_ approach as it is expressing the
fact that the list of function names is really a single thing being
returned \(albeit one that is not a simple value.\) It is not
anticipated that the performance of this function will need to be
crucial to too many applications.
# info functions
This new subcommand will provide access from Tcl scripts to the
functionality of _Tcl\_ListMathFuncs_. It will take a single optional
argument consisting of a pattern to pass on as the _pattern_ argument
\(with the absence of the argument indicating that NULL is to be passed
instead.\)
# Copyright
This document is placed in the public domain.

TIP: 150Title: Implement the Tk send Command for Windows
Author: Pat Thoyts <patthoyts@users.sourceforge.net>
Type: Project
State: Deferred
Vote: Done
Tcl-Version: 8.5
Version: $Revision: 1.6 $
Keywords: tk, send
Created: 25-Jul-2003
Post-History:
~ Abstract
This TIP proposes to provide an implementation of the send mechanism
for Tk under Windows.
~ Rationale
The Tk send command is used to provide an X server based mechanism for
transmitting information between Tk applications. This provides the
basis for remote introspection (tkinspect) and remote control (tkcon).
This subsystem has never been implemented under Windows although there
are means to emulate this using Tcl's dde package or the comm library.
By providing a win32 implementation we help to standardise the Tk
command set across platforms and will make it simpler to use this
feature from Perl/Tk and Python Tkinter.
~ Reference Implementation
This implementation is based upon an integrated version of the winsend
package currently at http://tclsoap.sf.net/winsend/. This uses the
Windows Running Object Table to maintain interpreter registration
information. This is a global system COM object that is provided as
part of the Windows COM automation framework. Its purpose is to
provide a single registry for maintaining references to active COM
components.
When ''tk appname'' is called a COM component is created which
contains a reference to the Tcl interpreter. This is then registered
with the Running Object Table with the registration name containing
the Tk app-name. To find available interpreters we use the ''winfo
interps'' command. This iterates over all the registered objects
looking for Tk registered names and can then return a list of
registered interpreters.
The Tk ''send'' command is then provided as a COM method call. To
perform a send, the client interpreter (or indeed any automation
capable COM client) obtains the interpreter reference from the running
object table and calls the ''Send'' method on this object, passing in
the command as a string. For the synchronous version this is
sufficient. The result is returned using the standard COM method
calling and error information can be returned the same way.
Support for the asynchronous send is currently underway but will be
modelled closely on the X-based asynchronous send.
................................................................................
never broadcasts. We initiate a conversation with the Running Object
Table first and then with the specific application requested. If this
application fails to process its message queue then we will get stuck
- but no other poorly behaved application can affect our
communication.
A second advantage is than non-Tcl automation capable clients can also
call our method. This means that Windows Scripting code (vbscript or
jscript) can make use of the send command.
Finally, given a suitable DCOM environment it should also be possible
to implement the -displayof feature via Distributed COM. However, I
don't see much call for this - network sockets are a lot simpler to
use than DCOM.
~ Copyright
This document is placed in the public domain.

# TIP 150: Implement the Tk send Command for Windows
Author: Pat Thoyts <patthoyts@users.sourceforge.net>
Type: Project
State: Deferred
Vote: Done
Tcl-Version: 8.5
Keywords: tk, send
Created: 25-Jul-2003
Post-History:
-----# Abstract
This TIP proposes to provide an implementation of the send mechanism
for Tk under Windows.
# Rationale
The Tk send command is used to provide an X server based mechanism for
transmitting information between Tk applications. This provides the
basis for remote introspection \(tkinspect\) and remote control \(tkcon\).
This subsystem has never been implemented under Windows although there
are means to emulate this using Tcl's dde package or the comm library.
By providing a win32 implementation we help to standardise the Tk
command set across platforms and will make it simpler to use this
feature from Perl/Tk and Python Tkinter.
# Reference Implementation
This implementation is based upon an integrated version of the winsend
package currently at <http://tclsoap.sf.net/winsend/.> This uses the
Windows Running Object Table to maintain interpreter registration
information. This is a global system COM object that is provided as
part of the Windows COM automation framework. Its purpose is to
provide a single registry for maintaining references to active COM
components.
When _tk appname_ is called a COM component is created which
contains a reference to the Tcl interpreter. This is then registered
with the Running Object Table with the registration name containing
the Tk app-name. To find available interpreters we use the _winfo
interps_ command. This iterates over all the registered objects
looking for Tk registered names and can then return a list of
registered interpreters.
The Tk _send_ command is then provided as a COM method call. To
perform a send, the client interpreter \(or indeed any automation
capable COM client\) obtains the interpreter reference from the running
object table and calls the _Send_ method on this object, passing in
the command as a string. For the synchronous version this is
sufficient. The result is returned using the standard COM method
calling and error information can be returned the same way.
Support for the asynchronous send is currently underway but will be
modelled closely on the X-based asynchronous send.
................................................................................
never broadcasts. We initiate a conversation with the Running Object
Table first and then with the specific application requested. If this
application fails to process its message queue then we will get stuck
- but no other poorly behaved application can affect our
communication.
A second advantage is than non-Tcl automation capable clients can also
call our method. This means that Windows Scripting code \(vbscript or
jscript\) can make use of the send command.
Finally, given a suitable DCOM environment it should also be possible
to implement the -displayof feature via Distributed COM. However, I
don't see much call for this - network sockets are a lot simpler to
use than DCOM.
# Copyright
This document is placed in the public domain.

TIP: 151Title: Remove -e: Command Line Option from tclsh and wish
Version: $Revision: 1.7 $
Author: Don Porter <dgp@users.sourceforge.net>
Author: Don Porter <dgp@users.sf.net>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 22-Aug-2003
Post-History:
Tcl-Version: 8.5
~ Abstract
This TIP proposes removal of the -e: command line option to
tclsh and wish that was Accepted as part of [137].
~ Background
[137] was Accepted today. However, there were a few NO votes
objecting to the new ''-e:'' form of the ''-encoding'' command
line option for specifying the encoding of a startup script to tclsh
and wish. Those voting NO only objected to that part of [137]
while supporting the rest as a solid proposal that will improve
Tcl. Among those voting YES, no one explicitly embraced the
''-e:'' command line option as something they required. Some
voting YES opined that the ''-e:'' option was a wart that
could be fixed later.
Based on those comments, [137] would have been better had the
''-e:'' form of the command line option not been part of the
proposal. This TIP proposes nothing more than removing
Acceptance of the ''-e:'' form of the ''-encoding'' command
line option.
Without the controversial ''-e:'' proposal, I believe [137]
would have had unanimous approval.
~ Rationale
The use of ''-e:'' as a command line option to tclsh or wish
suffers when compared with the perl program. The ''perl -e''
option for evaluation of a Perl script provided on the command
line is very well known, and it's a mistake to add something
to tclsh that looks similar, but is in fact very different.
[137] proposed both ''-encoding'' and the ''-e:'' form. There's
really no need to add multiple ways to do the same thing.
The Rationale in [137] for the ''-e:'' form is solely to support
the 32-character limit in some Unices for their ''#!'' lines.
However, the ''-e:'' form does not really solve that problem.
For example:
| #!/usr/local/bin/tclsh8.5 -e:iso8859-15
| 12345678901234567890123456789012
Here we see that we still run afoul of the 32-character limit
when tclsh is installed in the default location. Even longer
encoding names exist which magnify the problem, and altering the
installation location will not necessarily help:
| #!/usr/bin/tclsh8.5 -e:iso8859-15
| 12345678901234567890123456789012
(This is actually an insidious failure mode in that it leads to the
script being sourced with a valid but incorrect encoding.)
Conversely, we already have an effective general workaround
for the 32-character limit problem:
| #!/bin/sh
| # \
| exec tclsh -encoding iso8859-15 "$0" ${1+"$@"}
So, ''-e:'' doesn't solve a problem we don't really have, and
it's controverisal. We should remove it.
~ Proposal
Remove (Acceptance of) the ''-e:'' set of command line options
to the programs ''tclsh'' and ''wish''.
~ Compatibility
Since no version of Tcl or Tk has been released supporting
the ''-e:'' command line option, there are no compatibility
issues to resolve.
~ Copyright
This document is placed in the public domain.

# TIP 151: Remove -e: Command Line Option from tclsh and wish
Author: Don Porter <dgp@users.sourceforge.net>
Author: Don Porter <dgp@users.sf.net>
Author: Donal K. Fellows <donal.k.fellows@man.ac.uk>
State: Final
Type: Project
Vote: Done
Created: 22-Aug-2003
Post-History:
Tcl-Version: 8.5
-----# Abstract
This TIP proposes removal of the -e: command line option to
tclsh and wish that was Accepted as part of [[137]](137.md).
# Background
[[137]](137.md) was Accepted today. However, there were a few NO votes
objecting to the new _-e:_ form of the _-encoding_ command
line option for specifying the encoding of a startup script to tclsh
and wish. Those voting NO only objected to that part of [[137]](137.md)
while supporting the rest as a solid proposal that will improve
Tcl. Among those voting YES, no one explicitly embraced the
_-e:_ command line option as something they required. Some
voting YES opined that the _-e:_ option was a wart that
could be fixed later.
Based on those comments, [[137]](137.md) would have been better had the
_-e:_ form of the command line option not been part of the
proposal. This TIP proposes nothing more than removing
Acceptance of the _-e:_ form of the _-encoding_ command
line option.
Without the controversial _-e:_ proposal, I believe [[137]](137.md)
would have had unanimous approval.
# Rationale
The use of _-e:_ as a command line option to tclsh or wish
suffers when compared with the perl program. The _perl -e_
option for evaluation of a Perl script provided on the command
line is very well known, and it's a mistake to add something
to tclsh that looks similar, but is in fact very different.
[[137]](137.md) proposed both _-encoding_ and the _-e:_ form. There's
really no need to add multiple ways to do the same thing.
The Rationale in [[137]](137.md) for the _-e:_ form is solely to support
the 32-character limit in some Unices for their _\#!_ lines.
However, the _-e:_ form does not really solve that problem.
For example:
#!/usr/local/bin/tclsh8.5 -e:iso8859-15
12345678901234567890123456789012
Here we see that we still run afoul of the 32-character limit
when tclsh is installed in the default location. Even longer
encoding names exist which magnify the problem, and altering the
installation location will not necessarily help:
#!/usr/bin/tclsh8.5 -e:iso8859-15
12345678901234567890123456789012
\(This is actually an insidious failure mode in that it leads to the
script being sourced with a valid but incorrect encoding.\)
Conversely, we already have an effective general workaround
for the 32-character limit problem:
#!/bin/sh
# \
exec tclsh -encoding iso8859-15 "$0" ${1+"$@"}
So, _-e:_ doesn't solve a problem we don't really have, and
it's controverisal. We should remove it.
# Proposal
Remove \(Acceptance of\) the _-e:_ set of command line options
to the programs _tclsh_ and _wish_.
# Compatibility
Since no version of Tcl or Tk has been released supporting
the _-e:_ command line option, there are no compatibility
issues to resolve.
# Copyright
This document is placed in the public domain.

TIP: 152Title: New -detail Option for tk_messageBox
Version: $Revision: 1.7 $
Author: Mats Bengtsson <matben@privat.utfors.se>
State: Final
Type: Project
Tcl-Version: 8.5
Vote: Done
Created: 26-Aug-2003
Obsoletes: 25
Post-History:
Keywords: Tk
~ Abstract
This TIP proposes a new option for the '''tk_messageBox''' for text
that is less significant than the '''-message''' text.
~ Rationale
Several platforms have a native feature of alert dialogs that includes
messages with both a bold or large font, and a text in fine print,
which gives more detail to the actual message. Both Mac OS 8/9 and
Mac OS X include such a feature. This TIP suggests that this should
be intruduced on all platforms, with the text to be displayed in that
format specified using a new option '''-detail'''. In those cases where
it is not straightforward to distinguish between the '''-message''' and
'''-detail''' option contents, they should be combined into a single piece
of text in a way that makes sense on the platform.
~ Reference Implementation
The proposed change is now implemented as a loadable extension (in C)
on Mac OS X
[http://hem.fyristorg.com/matben/download/MovableAlerts.dmg].
The following images show the present (8.4.4) message box on AquaTk,
and the proposed one.
#image:152before This is the present tk_messageBox.
#image:152after This is the native tk_messageBox.
~ Copyright
This document has been placed in the public domain

# TIP 152: New -detail Option for tk_messageBox
Author: Mats Bengtsson <matben@privat.utfors.se>
State: Final
Type: Project
Tcl-Version: 8.5
Vote: Done
Created: 26-Aug-2003
Obsoletes: 25
Post-History:
Keywords: Tk
-----# Abstract
This TIP proposes a new option for the **tk\_messageBox** for text
that is less significant than the **-message** text.
# Rationale
Several platforms have a native feature of alert dialogs that includes
messages with both a bold or large font, and a text in fine print,
which gives more detail to the actual message. Both Mac OS 8/9 and
Mac OS X include such a feature. This TIP suggests that this should
be intruduced on all platforms, with the text to be displayed in that
format specified using a new option **-detail**. In those cases where
it is not straightforward to distinguish between the **-message** and
**-detail** option contents, they should be combined into a single piece
of text in a way that makes sense on the platform.
# Reference Implementation
The proposed change is now implemented as a loadable extension \(in C\)
on Mac OS X
<http://hem.fyristorg.com/matben/download/MovableAlerts.dmg> .
The following images show the present \(8.4.4\) message box on AquaTk,
and the proposed one.
![This is the present tk_messageBox.](../assets/152before.gif)![{} This is the native tk_messageBox.](../assets/152after.gif)# Copyright
This document has been placed in the public domain

TIP: 153Title: Enhancing the [winfo toplevel] Command
Version: $Revision: 1.8 $
Author: Neil McKay <mckay@eecs.umich.edu>
State: Final
Type: Project
Vote: Done
Created: 26-Aug-2003
Post-History:
Tcl-Version: 8.5
~ Abstract
The [[winfo toplevel]] command returns the Tk toplevel window that
encloses the window that's passed as an argument. However, Tk
extensions may allow the creation of windows which reside at the top
of a window hierarchy, but are not Tk toplevel widgets. If a
subwindow of one of these top-of-hierarchy widgets is passed to
[[winfo toplevel]], it returns an empty string, making it impossible
to determine what hierarchy the window resides in. This TIP proposes
enhancing the [[winfo toplevel]] command so that it will return the top
window in the hierarchy regardless of what type of widget it is.
~Rationale
Focus control does not work properly for top-of-hierarchy Tk windows
other than toplevel widgets. This is an artifact of Tk's inability to
determine the top of a window hierarchy for such windows. Prior to Tk
8.4, top-of-hierarchy windows other than toplevels were not possible,
so the changes proposed in this TIP would be moot. However, they are
now possible due to the incorporation into the Tk core of the changes
proposed in [47].
As an example, consider the rootwin extension
[http://www.eecs.umich.edu/~mckay/computer/rootwin0.4.tar.gz].
This extension adds a [[rootwin]] widget creation command,
which turns an X display's root window into a Tk widget. The rootwin
widget is a top-of-hierarchy widget, but is not an ordinary Tk toplevel.
The command sequence
| rootwin .r
| entry .r.e
| place .r.e -x 100 -y 100
will create an entry widget near the upper left corner of the screen.
However, it is impossible to type anything into the entry widget,
because Tk's focus code does not work properly with rootwin widgets.
The problem can be traced back to the fact that the command
[[winfo toplevel .r.e]] returns the empty string. With the proposed
extension, this command would return ".r", and the focus code
would work properly.
~Compatibility
The proposed behavior for [[winfo toplevel]] is different from the
current behavior, although only in the presence of Tk extensions
that create top-of-hierarchy widgets that are not toplevels.
The rootwin package is the only such extension known to this TIP's
author, so the difference in behavior is unlikely to be noticed
by anyone else. It should also be noted that the behavior
(prior to this TIP) of [[winfo toplevel]] for
top-of-hierarchy widgets that are not toplevels is not specified,
so relying on its current behavior in such cicumstances is
a questionable practice.
~Implementation
The core of the proposed change to [[winfo toplevel]] requires
only a single-line change in file ''generic/tkWindow.c''; however, the patch
described below also changes the name of a function and changes
some comments, to better describe the code's behavior. It also
documents the new behavior in the [[winfo]] man page.
~Patches
A patch (against Tk 8.4.5) that implements the changes described above
may be found at
http://www.eecs.umich.edu/~mckay/computer/winfotop.patch
~Copyright
This document is in the public domain.

# TIP 153: Enhancing the [winfo toplevel] Command
Author: Neil McKay <mckay@eecs.umich.edu>
State: Final
Type: Project
Vote: Done
Created: 26-Aug-2003
Post-History:
Tcl-Version: 8.5
-----# Abstract
The [winfo toplevel] command returns the Tk toplevel window that
encloses the window that's passed as an argument. However, Tk
extensions may allow the creation of windows which reside at the top
of a window hierarchy, but are not Tk toplevel widgets. If a
subwindow of one of these top-of-hierarchy widgets is passed to
[winfo toplevel], it returns an empty string, making it impossible
to determine what hierarchy the window resides in. This TIP proposes
enhancing the [winfo toplevel] command so that it will return the top
window in the hierarchy regardless of what type of widget it is.
# Rationale
Focus control does not work properly for top-of-hierarchy Tk windows
other than toplevel widgets. This is an artifact of Tk's inability to
determine the top of a window hierarchy for such windows. Prior to Tk
8.4, top-of-hierarchy windows other than toplevels were not possible,
so the changes proposed in this TIP would be moot. However, they are
now possible due to the incorporation into the Tk core of the changes
proposed in [[47]](47.md).
As an example, consider the rootwin extension
<http://www.eecs.umich.edu/~mckay/computer/rootwin0.4.tar.gz> .
This extension adds a [rootwin] widget creation command,
which turns an X display's root window into a Tk widget. The rootwin
widget is a top-of-hierarchy widget, but is not an ordinary Tk toplevel.
The command sequence
rootwin .r
entry .r.e
place .r.e -x 100 -y 100
will create an entry widget near the upper left corner of the screen.
However, it is impossible to type anything into the entry widget,
because Tk's focus code does not work properly with rootwin widgets.
The problem can be traced back to the fact that the command
[winfo toplevel .r.e] returns the empty string. With the proposed
extension, this command would return ".r", and the focus code
would work properly.
# Compatibility
The proposed behavior for [winfo toplevel] is different from the
current behavior, although only in the presence of Tk extensions
that create top-of-hierarchy widgets that are not toplevels.
The rootwin package is the only such extension known to this TIP's
author, so the difference in behavior is unlikely to be noticed
by anyone else. It should also be noted that the behavior
\(prior to this TIP\) of [winfo toplevel] for
top-of-hierarchy widgets that are not toplevels is not specified,
so relying on its current behavior in such cicumstances is
a questionable practice.
# Implementation
The core of the proposed change to [winfo toplevel] requires
only a single-line change in file _generic/tkWindow.c_; however, the patch
described below also changes the name of a function and changes
some comments, to better describe the code's behavior. It also
documents the new behavior in the [winfo] man page.
# Patches
A patch \(against Tk 8.4.5\) that implements the changes described above
may be found at
<http://www.eecs.umich.edu/~mckay/computer/winfotop.patch># Copyright
This document is in the public domain.

TIP: 154Title: Add Named Colors to Tk
Version: $Revision: 1.26 $
Author: Damon Courtney <damon@unreality.com>
State: Draft
Type: Project
Vote: Pending
Created: 03-Sep-2003
Post-History:
Tcl-Version: 8.7
~ Abstract
This TIP proposes the addition of a '''color''' command at the Tk level to
allow developers to create named colors as they can already do with both fonts
and images.
~ Rationale
Named fonts and images go a long way toward making Tk able to modify a lot of
options on a global scale simply by adjusting a single item. Tk already
simulates some version of named colors on Windows with the use of values like
''SystemButtonFace'' and ''SystemWindowFrame''. These color names map to
values pre-defined in Windows. On UNIX, we have to kind of fake these things
around.
With the addition of named colors, we can simply define default colors in UNIX
that correspond to existing named colors on Windows. So, we could create
''SystemButtonFace'' as a named color on UNIX, and any Windows program using
those color names would port to UNIX without a change.
~ Implementation
I propose the introduction of a new command, '''color''':
> '''color''' ''option'' ?''arg ...''?
This will have six subcommands.
The '''create''' subcommand creates a new named color.
> '''color create''' ''colorName'' ?''option ...''?
Supported options (see below for full description) are:
-color: The main color associated with this named color.
-dark1: The low-light color associated with this named color.
-light1: The high-light color associated with this named color.
The '''cget''' subcommand retrieves an option associated with a named color
> '''color cget''' ''colorName option''
The '''configure''' subcommand sets an option (or options) associted with a
named color, or retrieves information about all the options associated with
that named color (following the standard Tk pattern for cget/configure.)
> '''color configure''' ''colorName'' ?''option ...''?
The '''exists''' subcommand determines if a named color exists.
> '''color exists''' ''colorName''
The '''names''' subcommand returns a list of all named colors, including those
created by Tk. The optional glob-style ''pattern'' allows only some colors to
be returned.
> '''color names''' ?''pattern''?
The '''delete''' subcommand deletes a named color.
> '''color delete''' ''colorName''~~Options
The '''-color''' option specifies any acceptable color in Tk as the color to
use. The '''-light1''' and '''-dark1''' options specify the border shadings to
use for this color when it is used as a border. They are named with a 1
because I think in the future we will end up with more levels of shading.
Windows already has two levels, we just ignore the second one.
Beyond the color command, we need to implement a default set of standard
colors that will exist across all platforms. As proposed by some, I think
these should be named Tk*. The current list (reflected in the Windows color
list) would be:
|TkButtonFace
|TkButtonText
|TkDisabledText
|TkHighlight
|TkHighlightText
|TkMenu
|TkMenuText
|TkScrollbar
|TkWindow
|TkWindowFrame