Talk presented by C. Ballance at the 27th ICPEAC, Queen's University Belfast, 27/07-02/08 2011.

1. An R-matrix approach for plasma
modelling and the interpretation of
astrophysical plasmas
th
July 27 , Queen's University, ICPEAC 2011
Connor Ballance
Auburn University
Collaborators : T G Lee, S D Loch, M S Pindzola (AU)
: N R Badnell ( Strathclyde )
: B M McLaughlin (QUB)
: M A Bautista (WMU)
Supported by : US DoE Fusion Energy Sciences

2. Overview
● Introduction : Comprehensive approach to plasma
modelling, our current capabilities
and future directions.
● Electron-impact ionisation : high n shell ionisation
● Electron-impact excitation : scripted R-matrix calculations
: parallel DARC code
: Fe-Peak elements and beyond
● Photo-ionisation of Mid-Z elements : parallel dipole (DARC)
codes

3. Introduction
● In recent years, EIE R-matrix calculations, have moved beyond
the isolated, one-off serial calculations to parallel calculations
along entire iso-nuclear/iso-electronic sequences
Witthoeft et al 2007 (J. Phys. B Vol 40)
Liang and Badnell 2010 (Astron. Astrophys. Vol 518 A64)
Perl-scripted calculations, automatically calculate tabulated
every effective collision strength for all transitions
from user given structure.
● This data is stored in a well-prescribed format that includes
the atomic configurations, the energy levels, the A-Values
for all E1,E2,M1,M2 transitions, Maxwellian averaged
collision strengths for a range of temperatures and the
Born/Bethe infinite energy limit points. (why?)
.

4. http://www-cfadc.phy.ornl.gov/home.html

5. Introduction
● Modern computing archictectures have over 100,000
cpu cores
Kraken, NICS Oak Ridge, TN (Cray )
Hopper, NERSC, Berkley, CA (Cray)
and if utilised correctly, can support PetaFlop/sec calculations.
● The serial codes (1973-1999) could take a week(s)
to calculate an ion stage. The first generation of parallel
codes (2-3 hundred levels, 1-2 thousand channels) a day
● Now we need adapt to (1-3 thousand level calculations
with 5-20 thousand channels) if we are to address open
p-and-d shell systems.

6. R-matrix/R-matrix with
Pseudostates (RMPS) review

7. The importance of excited state
ionisation
Effective ionisation rates include the contribution of excited state ionisation,
●
which becomes computationally demanding for non-perturbative methods
such the RMPS
Allain et al 2004
Nucl. Fusion 44, 655 (2004)
How do we employ the RMPS
method effectively, for high
n shell ionisation ?
The calculation of ionisation
from every term of a high
n can be an order of magnitude
more computationally demanding
than the groundstate alone.

8. There are many non-perturbative
ionisation cross-sections from the
groundstate
For example, consider the closed shell groundstate of neutral neon
Groundstate Excited terms

9. However systems with several valence electrons soon become problematic
Consider the all LS coupling electron-impact ionisation (both ground and excited) state
from a boron-like system such as B I / C II
This will require ionisation from : 1s^2 2s^2 nl (where n=2-4, l=0-3)
: 1s^2 2s 2p^2
: 1s^2 2p^3 (for C II )
In addition to the above spectroscopic terms, we shall require minimum pseudostate
expansions of the form:
1. 1s^2 2s^2 nl (where n=5,14,l=0-6)
2. 1s^2 2s 2p nl (where n=5,14,l=0-6)
3. 1s^2 2p^2 nl (where n=5,14,l=0-6)
If you want to calculate
a) Direct ionisation of the outer shell electron
b) Direct ionisation of the 2s electron
c) All the excitation-autoionisations from every term ie. e + 1s^2 2s^2 3s
--> 1s^2 2s2p 3s
Well, 1444 terms , approximately 5000 close-coupled channels and 5 Tb of
Hamiltonian matrices requiring diagonalisation poses an interesting challenge ...

10. This represents the current work, that extends beyond naively splitting the
serial problem over more processors, to one in which the parallel code
adapts to a particular problem
Hamiltonian formation
1. Serial : Each partial wave is calculated consecutively ( 50-100 ) .... a month
2. Naive parallelism : each partial wave is carried out concurrently .... 3 days
(remember a single partial wave > 200 Gbs) .... 100 procs
3. Adaptive parallelism : As well as each partial wave being carried out concurrently
the target terms are grouped into their L S Pi groups
(perhaps 20-40 unique groups) .... 2000-4000 procs … 4hrs
RESULT : Hamiltonian formation is reduced scattering from a set of target terms with the
same L S Pi values.
Hamiltonian diagonalisation
1.Serial : Impossible ! Every eigenvalue of a 200 K by 200 K Hamiltonian
2. Naive parallelism : sequential parallel diagonalisation using Scalapack, possible,
but regardless of diagonalisation time, you must read 5 Tb .... 4 days
3.Adaptive parallelism : Each Hamiltonian is concurrently diagonalised in parallel , with
an n^3 scaling law controlling the distribution of processors … 5 hrs

11. RESULT : Adaptive diagonalisation ---> 1 Hamiltonian read, 1 diagonalisation
Better load balancing as processing power is distributed to where it is needed

12. Electron-impact ionisation of neutral Boron , n=3 shell
Notice: large excitation-autoionisation (EA) i.e. e + 1s^2 3l --> 1s^2 2s 2p 3l + e
3d
3p
3s
Large EA destroys any n^4 scaling law (or rescaling of an empirical formula) needed
to extrapolate to higher n shell ..... solution ?

13. If we can explicitly caculate to high enough n shell, that the direct ionisation
completely dominates over EA , then we can scale simple ionisation expressions from
the last explicitly calculated n shell shell of the RMPS
Below , we have statistically averaged the term resolved RMPS results used
to rescale an expression, such as the Burgess-Vriens for the higher n shells.
n=4
B
+
B
n=4
2+
B n=5

14. Excited state ionisation from
higher charged ions
Higher partial waves begin to
dominate the cross section, for
high n , multiply charged systems
Convergence is very slow
Comparisons with distorted
wave are favourable, again
RMPS becomes computational
Demanding, requiring
110-130 pseudo-orbitals
ranging from n=6-16,18 and l=0-10
to achieve convergence
Time : distorted wave (15 mins)
RMPS (4 hrs)
Distorted wave (Younger Potential)
Distorted Wave (Macek Potential)
RMPS

15. Electron-impact excitation
and how we adapt to future
challenges.

16. Scripted Semi-relativistic ICFT (Intermediate Coupling Frame Transformation)
R-matrix calculations
● Intermediate Coupling Frame Transformation, developed by D C Griffin is an
R-matrix approach primarily carried out in LS coupling but including mass-velocity
and Darwin terms, that transforms term-resolved K-matrices into level-level
K-matrices, and ultimately level resolved excitation rates. Provides comparable
cross sections to the Breit-Pauli R-matrix codes.
● A perl-script has been developed that automatically takes structure calculations
for atomic ions Z < 36 , through to effective collision strengths for ALL ions along
an iso-electronic sequence
Example Papers:
G. Y. Liang et al Astron. Astrophys. 528 A69(15) (2011). Li-like sequence
G. Y. Liang et al Astron. Astrophys. 518 A64(20) (2010). Neon sequence
G. Y. Liang et al J. Phys. B 42 225002(12) (2009). Na-like sequence
M. C. Witthoeft et al J. Phys. B 40 2969-93 (2007). Fl-like sequence
Typically 100-300 levels calculations, with 1000-1500 channels

17. However, eventually as Z increases we must adopt the
a parallel version of relativistic R-matrix codes.
DARC (Dirac Atomic R-matrix Code)
● These codes have been modified in an analguous way to the non-relativistic codes
- distribution of integral generation
- multi-layered parallelism of the Hamiltonian formation
- Concurrent adaptive diagonalisation of the Hamiltonian
Excitation/ionisation
of Mo II
0.6 billion Racah coefficients
per symmetry 4d^5,4d^4 4f-5f
This is preparation
8000-10000 channels calculation for future
still a challenge W ion stages.

18. e.g. DARC calculations for Fe III
Target :
3d^6, 3d^54s, 3d^54p, 3p^43d^8, 3p^53d^7 (2-3 hundred level)
ideally now we would add (3p^54d, 3p^43d^74s ,3p^43d^74p)
which if we keep to 0.0-0.5 Ryds, provides a satisfactory description of Fe III

19. e.g. DARC calculations for Fe II
Before After
(Ith -Iobs)/Iobs
Collision strengths : Zhang 1992 Collision stengths : parallel DARC
A-values : Nahar and Pradhan A-values : HFR (cowan)

20. ICFT DARC
A typical collision strength within the groundstate complex of Fe III
illustrates the need for a very fine energy mesh !

21. However, if we want continued scaling to 5-12 thousand close-coupled
channels, reconsider electron-impact ionisation of B I
* 1444 terms
* 5000 channels ,
* Hamiltionian Matrices close to 200,000 by 200, 000 in size
* over 1 million possible transitions
But the formation of the R-matrix is crippling in both memory and the time
required! .... cannot be left to a single processor
W ik * W kj 200 000 * 5000 * 8 * 2
R
ij = k
E-E = 16 Gb ! (takes mins )
k
0.2 Gb N slices
w ik / E - E k
w
= kj
5000*5000*8

23. 928/61440 = 0.015 sec per R-matrix formation

24. We can also adapt the distribution of processors to energy points in the outer region
Test run : 4 partial waves
Se III (225 levels): 437,737,748,1017 channels
12,000 pts
. CPU TIME= 1.380 MIN -- processors=: 250 sub world 0
CPU TIME= 3.851 MIN -- processors=: 250 sub world 1
CPU TIME= 3.990 MIN -- processors=: 250 sub world 2
CPU TIME= 6.496 MIN -- processors=: 250 sub world 3
Suggested Proc distribution: 89 245 253 413
CPU TIME= 3.563 MIN -- processors=: 245 sub world 1
CPU TIME= 3.868 MIN -- processors=: 413 sub world 3
CPU TIME= 3.869 MIN -- processors=: 89 sub world 0
CPU TIME= 3.977 MIN -- processors=: 253 sub world 2
-------------------------------------------------------------
Another 2.5 mins saved .... better balance of resources
RECAP

25. Photoionisation of Mid-Z atomic ions
Posters : Thu 140,Thu 143

26. An overview of the parallel DARC dipole photoionisation codes

27. An overview of the parallel DARC dipole photoionisation codes
● The parallel dipole suite of codes, benefit from the changes made from excitation
All the eigenvectors from a pair of dipole allowed symmetries are required
for bound-free dipole matrix formation
The current approach ensures that every dipole matrix pair is carried out concurrently
●
with groups of processors assigned to an individual dipole.
RESULT : ALL photoionisation, dielectronic-recombination or radiative-damped
excitation takes the time for a single dipole formation
● The capacity to perform photoionisations calculations with over 500 levels, improves
the residual ion structure, the ionisation potential and resonance structure associated
with over 3500 channels.
As the code scales to 100,000 processors , we can have a resolution of 10^(-8) Rydbergs
●
ie (6-30 million pt) in the incident photon energy, which is vital when comparing with
ALS measurements at 4,6,9 meV.
● Of course, theoretical calaculations can guide experimental measurement and
determine metastable fractions in experimental measurement.

28. Example : Groundstate Photoionisation of Ca II (work in progress)
Target : 3p^6,
3p^5[4s-5p]
3p^4 3d 4s hv + Ca II (^2 S) 3p^64s
3p^4 3d 4p
3P^4 3d 4d which gives rise to 513 levels
Experiment : Lyon (1987)
ALS(2010)
Theoretical : DARC R-matrix
Currently, the theoretical model
is only from the groundstate,
with metastable calculations
ongoing.

29. Example : Photoionisation of Kr II and Xe II
● Used a 326 level model for the residual ion in both cases
●
Target configurations for K III include : 4s^2 4p^4
4s 4p^5
4p^6
4s^2 4p^2 4d^2
4s^2 4p^3 4d
4s 4p^4 4d
4p^4 4d^2
● Target configurations for Xe III (exactly the same : switching n=4 to 5)
● In the following graphs the R-matrix results have been statistically averaged
over the initial p^5 levels

31. The agreement with Xe II is equally good

32. Come by the poster for more details … TH 143

33. Thank-you for your attention