I was wondering if there is any clever memory usage ways to store symmetric arrays. I.e. I might have some tensor f_[abcd] = f_[bacd] = .. etc i.e. with total permutational symmetry of the indicies. Storing every single element will require N! times the data that's actually included in the tensor, which with N=4 is already 24.
My current "solution" is to only populate the array for a <= b <= c <= d etc and put zeros for the rest and simply sort the arguments when I look up an element in this array. It is however not possible to have "sparse" arrays so these zeros, I assume, still take up some space in the memory.
I'm wondering if anyone has a clever solution to my problem?

"David Holdaway" <ddhwy@hotmail.com> wrote in message <ilqi9p$s4f$1@ginger.mathworks.com>...
> I was wondering if there is any clever memory usage ways to store symmetric arrays. I.e. I might have some tensor f_[abcd] = f_[bacd] = .. etc i.e. with total permutational symmetry of the indicies. Storing every single element will require N! times the data that's actually included in the tensor, which with N=4 is already 24.
> My current "solution" is to only populate the array for a <= b <= c <= d etc and put zeros for the rest and simply sort the arguments when I look up an element in this array. It is however not possible to have "sparse" arrays so these zeros, I assume, still take up some space in the memory.
> I'm wondering if anyone has a clever solution to my problem?
>
> Many thanks
- - - - - - - - - - -
There is a way you can accomplish the "compression" of your N-dimensional symmetric arrays, David. For example for N = 4 with an n by n by n by n symmetric array the total number of index combinations where i4 <= i3 <= i2 <= i1 is m = n*(n+1)*(n+2)*(n+3)/24. The following formula defines a one-to-one mapping from all such ordered index combinations onto the successive integers 1:m.

If you leave off the last line, the formula applies to the N = 3 case with ordered i3, i2, i1, and if the last two lines are omitted, it applies to the N = 2 case with just i2 and i1. Using such a formula is a little like using matlab's 'sub2ind' function except that it is more complicated.

The procedure for accessing an element with indices p, q, r, s would be to first sort them, as you have described, to a set i4<=i3<=i2<=i1 and then compute k as above, and then access the k-th element of a one-dimensional vector. One and only one value of k will be associated with each possible set of ordered indices. As you have pointed out, there is a saving with N = 4 of almost a factor of 24 to 1.

Admittedly as N gets larger the amount of computation increases for this transformation, but that seems to be the price one must pay for such a compression.

I have only worked this out up to N = 4. I perceive a pattern to this formula but it does not appear easy as yet to fully generalize it for an arbitrary N. One thing is clear - the expression is going to increase in complexity approximately as the square of N as N increases.

>
> I have only worked this out up to N = 4. I perceive a pattern to this formula but it does not appear easy as yet to fully generalize it for an arbitrary N. One thing is clear - the expression is going to increase in complexity approximately as the square of N as N increases.

After a very quick though, my impression is that the linear index is multiviate polynomial of the subindex. The polynomial is of degree N (somehow it recalls me Horner's rule to evaluate polynomial). Thus I doubt there is any close invert formula for N >= 5 Roger.

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <ilscso$8v8$1@ginger.mathworks.com>...
> After a very quick though, my impression is that the linear index is multiviate polynomial of the subindex. The polynomial is of degree N (somehow it recalls me Horner's rule to evaluate polynomial). Thus I doubt there is any close invert formula for N >= 5 Roger.
>
> Bruno
- - - - - - - -
Yes, performing the inverse to this transformation might be a difficult task. I concentrated here in only the subindices-to-linear index direction. That was messy enough for one day. It is not clear to me that David had anything this complicated in mind for his "compression". That is why I found it easy to stop at N = 4, even in that direction.

"Roger Stafford" wrote in message <ilsaus$4cg$1@ginger.mathworks.com>...
> "David Holdaway" <ddhwy@hotmail.com> wrote in message <ilqi9p$s4f$1@ginger.mathworks.com>...
> > I was wondering if there is any clever memory usage ways to store symmetric arrays. I.e. I might have some tensor f_[abcd] = f_[bacd] = .. etc i.e. with total permutational symmetry of the indicies. Storing every single element will require N! times the data that's actually included in the tensor, which with N=4 is already 24.
> > My current "solution" is to only populate the array for a <= b <= c <= d etc and put zeros for the rest and simply sort the arguments when I look up an element in this array. It is however not possible to have "sparse" arrays so these zeros, I assume, still take up some space in the memory.
> > I'm wondering if anyone has a clever solution to my problem?
> >
> > Many thanks
> - - - - - - - - - - -
> There is a way you can accomplish the "compression" of your N-dimensional symmetric arrays, David. For example for N = 4 with an n by n by n by n symmetric array the total number of index combinations where i4 <= i3 <= i2 <= i1 is m = n*(n+1)*(n+2)*(n+3)/24. The following formula defines a one-to-one mapping from all such ordered index combinations onto the successive integers 1:m.
>
> k = i1 + ...
> (i2-1)*(2*n-i2)/2 + ...
> (i3-1)*(3*n*(n+1)-(3*n+2)*i3+i3^2)/6 + ...
> (i4-1)*(4*n*(n+1)*(n+2)-(6*n^2+14*n+6)*i4+(4*n+5)*i4^2-i4^3)/24;
>
> If you leave off the last line, the formula applies to the N = 3 case with ordered i3, i2, i1, and if the last two lines are omitted, it applies to the N = 2 case with just i2 and i1. Using such a formula is a little like using matlab's 'sub2ind' function except that it is more complicated.
>
> The procedure for accessing an element with indices p, q, r, s would be to first sort them, as you have described, to a set i4<=i3<=i2<=i1 and then compute k as above, and then access the k-th element of a one-dimensional vector. One and only one value of k will be associated with each possible set of ordered indices. As you have pointed out, there is a saving with N = 4 of almost a factor of 24 to 1.
>
> Admittedly as N gets larger the amount of computation increases for this transformation, but that seems to be the price one must pay for such a compression.
>
> I have only worked this out up to N = 4. I perceive a pattern to this formula but it does not appear easy as yet to fully generalize it for an arbitrary N. One thing is clear - the expression is going to increase in complexity approximately as the square of N as N increases.
>
> Roger Stafford
- - - - - - - - -
David, by a simple rearrangement of the mapping I described yesterday, the transformation takes on a much simpler form. The formula below is for N = 8. It is now obvious how this can be generalized to any N. Notice that the value n, the size of each of the N dimensions, does not appear in the formula. It remains a bijective (one-to-one) mapping.

Bruno, in this simpler form it is now clear that the problem of performing the inverse of this mapping is dependent on finding the inverse of polynomial functions of the form (x-1)*x*(x+1)*(x+2)*(x+3)*... For x >= 1 the inverse is unique. I wonder if anyone has ever worked out a good algorithm for this that would be practical in this problem. I suppose one could use 'roots' and always select the real root greater than or equal to 1.

"Roger Stafford" wrote in message <ilu9dk$4o6$1@ginger.mathworks.com>...

>
> Bruno, in this simpler form it is now clear that the problem of performing the inverse of this mapping is dependent on finding the inverse of polynomial functions of the form (x-1)*x*(x+1)*(x+2)*(x+3)*... For x >= 1 the inverse is unique. I wonder if anyone has ever worked out a good algorithm for this that would be practical in this problem. I suppose one could use 'roots' and always select the real root greater than or equal to 1.

Thanks Roger to derive the formula (I was sure that you are succeeded in such task). I suppose there is something more clever than calling ROOTS since the indices are integer and the progression is quite predictable. I'm thinking along a dichotomy or golden-search. Not sure, but could it be possible to relate the question to gamma and inverse functions and family?

What is a watch list?

You can think of your watch list as threads that you have bookmarked.

You can add tags, authors, threads, and even search results to your watch list. This way you can easily keep track of topics that you're interested in. To view your watch list, click on the "My Newsreader" link.

To add items to your watch list, click the "add to watch list" link at the bottom of any page.

How do I add an item to my watch list?

Search

To add search criteria to your watch list, search for the desired term in the search box. Click on the "Add this search to my watch list" link on the search results page.

You can also add a tag to your watch list by searching for the tag with the directive "tag:tag_name" where tag_name is the name of the tag you would like to watch.

Author

To add an author to your watch list, go to the author's profile page and click on the "Add this author to my watch list" link at the top of the page. You can also add an author to your watch list by going to a thread that the author has posted to and clicking on the "Add this author to my watch list" link. You will be notified whenever the author makes a post.

Thread

To add a thread to your watch list, go to the thread page and click the "Add this thread to my watch list" link at the top of the page.

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

About Newsgroups, Newsreaders, and MATLAB Central

What are newsgroups?

The newsgroups are a worldwide forum that is open to everyone. Newsgroups are used to discuss a huge range of topics, make announcements, and trade files.

Discussions are threaded, or grouped in a way that allows you to read a posted message and all of its replies in chronological order. This makes it easy to follow the thread of the conversation, and to see what’s already been said before you post your own reply or make a new posting.

Newsgroup content is distributed by servers hosted by various organizations on the Internet. Messages are exchanged and managed using open-standard protocols. No single entity “owns” the newsgroups.

There are thousands of newsgroups, each addressing a single topic or area of interest. The MATLAB Central Newsreader posts and displays messages in the comp.soft-sys.matlab newsgroup.

How do I read or post to the newsgroups?

MATLAB Central

You can use the integrated newsreader at the MATLAB Central website to read and post messages in this newsgroup. MATLAB Central is hosted by MathWorks.

Messages posted through the MATLAB Central Newsreader are seen by everyone using the newsgroups, regardless of how they access the newsgroups. There are several advantages to using MATLAB Central.

Use the Email Address of Your Choice
The MATLAB Central Newsreader allows you to define an alternative email address as your posting address, avoiding clutter in your primary mailbox and reducing spam.

Spam Control
Most newsgroup spam is filtered out by the MATLAB Central Newsreader.

Tagging
Messages can be tagged with a relevant label by any signed-in user. Tags can be used as keywords to find particular files of interest, or as a way to categorize your bookmarked postings. You may choose to allow others to view your tags, and you can view or search others’ tags as well as those of the community at large. Tagging provides a way to see both the big trends and the smaller, more obscure ideas and applications.

Watch lists
Setting up watch lists allows you to be notified of updates made to postings selected by author, thread, or any search variable. Your watch list notifications can be sent by email (daily digest or immediate), displayed in My Newsreader, or sent via RSS feed.

Other ways to access the newsgroups

Use a newsreader through your school, employer, or internet service provider

Pay for newsgroup access from a commercial provider

Use Google Groups

Mathforum.org provides a newsreader with access to the comp.soft sys.matlab newsgroup