[Getdp] How many elements can GetDP handle?

[John_V]

How big of a problem is it reasonable to solve using GetDP? I.e., at 
what point (measured in number of equations or number of elements) does 
the time and/or memory requirement become too large to be practical?

I ask because I have made my first attempt to solve a real 3-D 
electrostatics problem, and I have discovered for myself that it is not 
easy to keep the memory usage within limits. The issue arises because of 
the different length scales in my problem.

I started with a much coarser mesh size than I'd like, just to see how 
things would go. I can solve a problem with the solver reporting N: 
33322. What is this? The number of equations? For a somewhat finer mesh 
(but still far from my goal) the solver reports N: 114379 and GetDP 
aborts with "PETSC ERROR: Out of memory." The message says the memory 
requested was 811961712. (I assume this is bytes, so about 812 MB.) This 
is a smaller limit than I expected--I expected to be able to use up to 
about 2 GB, the 32-bit Windows XP per-process limit. I am about to 
upgrade to a 64-bit machine with 12 GB memory, and I hope its larger 
addressable space will let me do larger problems, but if the limit is 
internal to GetDP rather than the hardware or operating system, the 
larger machine is unlikely to help.

I would also be interested in some more general ideas for how to deal 
with 3-D problems with varying length scales. I'm aware that I'm far 
from the first person to encounter this difficulty. What kinds of 
strategies to people use?

For concreteness: My sample has structures, mixed conductors and 
insulators, that are a few tens of nanometers in size with features 
(corner radii, edge widths) as small as 1 nm. These structures cover an 
area of up to 1 square micrometer. The insulators may have implanted 
charges down to a depth of up to a micrometer (though usually only a few 
tens of nanometers). Unfortunately, in general the sample need not have 
any particular symmetry. I would like to calculate the electrostatic 
potential in the neighborhood of the sample with errors of no more than 
a few tenths of a volt. I think this means my smallest elements, the 
ones near features of interest (edges, corners, maybe surfaces) should 
be sized close to 1 nm. Added to this, I'm presently meshing a volume 
that starts at the deepest implanted charges and goes above the sample 
far enough to be a good approximation of "infinity" -- i.e., far enough 
away so the potential is approaching 0. (Is there, for example, some 
nice trick to put that surface at "infinity" much closer without too 
strongly perturbing the solution near the sample surface?) Even if I 
choose big elements at the distant boundaries of my volume, the number 
of elements required is very large. It requires millions of elements 
just to cover the sample area one element deep.

John