In GAP, I tried for first group with p=3,( i.e. G=Z/3Z x Z/3Z): Z/3Z); but no output. It gives |Aut(G)|=432, but no structure description. So I am looking theoretically. I did following: since any automorphism of G fixes center(not necessarily pointwise), there is natural homomorphism f:Aut(G) --> Aut(G/Z(G)), where |Z(G)|=p, and G/Z(G)=Z/pZ x Z/pZ, Aut(G/Z(G))=GL(2,p). We can show that f is surjective with kernel Z/pZ x Z/pZ. Hence |Aut(G)|=(p^2).|GL(2,p)|=(p^3).(p+1).(p-1)^2. I couldn't proceed further.
–
RDKJan 21 '11 at 6:02

3

It's instructive to work out such examples from scratch, but it's also important to realize that this has certainly been done before and is most likely written down in detail (somewhere). With over a century of literature on group theory including many books, it could take a while to locate such information; maybe it's quicker to do it yourself. Anyway, it's also useful to organize the possible methods such as generators and relations, linear representations, geometric realizations. GAP has its limitations for giving insight.
–
Jim HumphreysFeb 22 '11 at 15:32

This example I couldn't see in any Graduate Studies book; it is not also given as an exercise. Late, I found that it is an exercise in the book "Structure of Groups of Prime Power Order". When working such examples in GAP, it gives the order of Automorphism group of these groups quickly, but it is not giving information about structure.
–
RDKFeb 23 '11 at 3:55

@Rahul: I deleted my answer (I think), and instead typed everything up into a PDF. You can email me and I'd be glad to send it to you.
–
Steve DFeb 23 '11 at 9:11

The former group can be seen as the group of unitriangular $3 \times 3$-matrices over the field with $p$ elements:
$$G = \left\{ \begin{pmatrix} 1 & * & * \\ 0 & 1 & * \\ 0 & 0 & 1 \end{pmatrix} \right\} \leq SL(3,p)$$
The automorphism groups of such groups have been studied (in a much larger generality); see for instance the paper "The automorphism group of the group of unitriangular matrices over a field" by Ayan Mahalanobis (http://arxiv.org/abs/1012.5534v1).