200. How long will your nwchem frequency calc take?

Update 19/12/12: Having done a lot more frequency calculations since I posted this I sincerely doubt that this approach works.

Original post
Because I'm stuck waiting for the results of frequency calcs on some large transition metal clusters, I've become interested in understanding the output of frequency calculations in progress. After all, why wait 15 days for a results if there are early signs that the calculation has gone haywire?

Also, it might just be me, but frequency calculations are not that easy to restart, so you want to make sure that you give them enough wall time to finish if you use a queue manager.

I'm sure most of this could be appreciated by RTFM, but who has time for that?

So this is what the calc does:
After the usual boredom of reading in the geometry and doing an energy calculation, followed by an MO analysis, the computational fun starts.

Each cycle contains the following reports:

1. Total Density - Mulliken Population Analysis
2. Spin Density - Mulliken Population Analysis
3. Total Density - Lowdin Population Analysis
4. Spin Density - Lowdin Population Analysis
5. Expectation value of S2:  
6. NWChem DFT Module
7.   Caching 1-el integrals 
8.       Total Density - Mulliken Population Analysis
9.       Spin Density - Mulliken Population Analysis

with the exception of the first cycle, which also look at the alpha-beta orbital overlaps, the centre of mass, moments of inertia and does a multipole analysis of density and save an initial hessian.

Each cycle ends with a report of the energy for that vibration:

         Total DFT energy =    -3297.032399945703

      One electron energy =   -26618.764098759657

           Coulomb energy =    12938.745973154924

    Exchange-Corr. energy =     -382.742230868704

 Nuclear repulsion energy =    10765.727956527733

 Numeric. integr. density =      441.999974968347

     Total iterative time =   7947.4s

If you do cat nwch.nwout|egrep "Total iterative time|Total DFT energy" you can see the progress:

        Total DFT energy =    -3297.032416366805
     Total iterative time =  12146.0s
         Total DFT energy =    -3297.032399945703
     Total iterative time =   7947.4s
         Total DFT energy =    -3297.032399544749
     Total iterative time =   7946.0s
         Total DFT energy =    -3297.032406934719
     Total iterative time =   7945.8s
         Total DFT energy =    -3297.032405026814

You now have an idea of how long each step takes. But how many steps in total? I think it's 3N steps, where N is the number of atoms.

For my 50 atoms POM using the values above it'd be roughly 8000 s * 150 = 13 days 22 hours.

Which seems about right...

cat nwch.nwout|grep 'Total DFT'|gawk 'END {print NR}'

66

so I've got another 8 days before I can get my hand on some juicy thermochemical data...

Time to start preparing lectures...

199. NeCTAR -Virtualisation of Australian compute resources -- first steps

So they are seeing whether they can make more efficient use of the compute resources at different institutions in Australia by creating a cloud to pool their resources. One of the potential solutions is called NeCTAR.

---

Getting started
Go to http://dashboard.rc.nectar.org.au/auth/login/?next=/dash/
Log in using your institutions username and password

You now have two options to deal with the key issue:

Method 1 -- generate online
Once you're in, create a keypair under Manage Compute/Access & Security and give it an easy-to-remember name

This is your private key, so protect it: don't lose it and don't expose it. You can't download it again. You delete it, it's gone.

On your computer
mv ~/Downloads/nectar.pem ~/.ssh
chmod og-rwx ~/.ssh/nectar.pem
cp nectar.pem nectar
ssh-keygen -e -f nectar >nectar.pub

 ls nectar* -lah

-rw------- 1 me me 887 Jun 22 11:31 nectar
-rw------- 1 me me 887 Jun 22 11:28 nectar.pem
-rw-r--r-- 1 me me 335 Jun 22 11:31 nectar.pub

To use the key do

ssh -i nectar user@server

Method 2 -- BYOK
You're using linux -- you probably have your own key already.
Go to the Manage Compute/Access & Security, Import Keypair

Paste your ~/.ssh/id_rsa.pub (or id_dsa.pub) key.

And that's the extent of the setup.

---

Test run
Go to Manage Compute/Images & Snapshot and select a Real Linux image (i.e. Debian)

Select image

Hit Launch.

Set up -- don't forget to check SSH to be able to log on. If you want to be able to ping, check icmp as well.

Set up the image -- the defaults are ok, but make sure to check icmp (to be able to ping) and ssh (to be able to log in).

Loading

Generating and loading the image takes about 10-20 seconds -- about the duration of a Victorian earthquake.

Running

Now your image is up an running. To check that all is well

ping -c 3 115.146.92.154

PING 115.146.92.154 (115.146.92.154) 56(84) bytes of data.
64 bytes from 115.146.92.154: icmp_req=1 ttl=54 time=1.70 ms
64 bytes from 115.146.92.154: icmp_req=2 ttl=54 time=1.66 ms
64 bytes from 115.146.92.154: icmp_req=3 ttl=54 time=1.73 ms

--- 115.146.92.154 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 1.660/1.698/1.733/0.029 ms

To be able to log in via ssh you need to know what username to use -- it's (probably) image specific.

To find out, click on the image name (here: testing4)

Click me

Hit the 'Log' tab

Select 'log'

 And look for the username which is created in addition to root

Look for the username -- here it's debian

ssh -v -i ~/.ssh/tmp/nectar debian@115.146.92.154
The authenticity of host '115.146.92.154 (115.146.92.154)' can't be established.
RSA key fingerprint is 81:a8:a7:0f:a9:68:a0:08:f1:60:45:e3:57:2e:4c:4c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '115.146.92.154' (RSA) to the list of known hosts.

Once you're in you'll be greeted by:

Linux unnamed-virtual-machine 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
debian@i-00001637:~$ 

debian@i-00001637:~$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/vda1             9.9G  768M  8.6G   9% /
tmpfs                 2.0G     0  2.0G   0% /lib/init/rw
udev                  2.0G  112K  2.0G   1% /dev
tmpfs                 2.0G     0  2.0G   0% /dev/shm
debian@i-00001637:~$ uname -a
Linux i-00001637 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012 x86_64 GNU/Linux
debian@i-00001637:~$ groups
debian cdrom floppy audio dip video plugdev
debian@i-00001637:~$ cat /etc/group|grep debian
cdrom:x:24:debian
floppy:x:25:debian
audio:x:29:debian
dip:x:30:debian
video:x:44:debian
plugdev:x:46:debian
debian:x:1000:

When you're done, don't forget to log out and terminate your image. If you leave it running it will count towards your resource allocations.

Terminating

---

Notes:

1. you'll run into trouble with the key fingerprints eventually as the IP addresses and key fingerprints won't be matching. Either you'll be doing a lot of editing of you ~/.ssh/known_hosts file or you have to relax your security setttings.

2. Yes, you can log in as root as well. The default user does not have sudo powers. 

3. It takes about 60 seconds after the launch of the image before the openssh server is up and accepting connections. Think more desktop speeds than laptop+SSD speeds.

4. For actual production stuff you can crank up the image requirements:

16 cores and 65 GB? Why, thank you!

5. Also, I think the real value of using virtual machine is that you can load a vanilla setup and customize it, then saving it by making a snapshot:

Snapshot saves

A first couple of actions might be to add a new user, and edit /etc/sudoers.

---

Troubleshooting ssh:
If you're having problems logging in using your key, use the ssh -v switch as shown above and parse the output



Unsuccessful:

debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/me/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/me/.ssh/id_dsa
debug1: Trying private key: /home/me/.ssh/id_ecdsa
debug1: Next authentication method: password

A successful authentication should contain

debug1: Offering RSA public key: /home/me/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug1: read PEM private key done: type RSA
debug1: Authentication succeeded (publickey).

If you are sure that you're using the right key (e.g. using -i), then make sure that you're using the right username -- to find out how to find it, look above.

198. Nwchem -- freeze core and tddft on benzene

UPDATE: it's taken a while, but I've tested this on a large polyoxometalate cluster (>10 group 5/6 atoms and >30 oxygens) -- with a total of ca 700 alpha and beta orbitals, respectively, I froze 200 core orbs and used 3-21G/3-21G* w/ ub3lyp. The frozen core calculation took 44% of the time the full calculation took. The spectra look identical to within 3 nm (18 roots -- only 2 intermediate transitions have been shifted. All other transitions are identical). The full calc took 7 days and 4 hours, while the frozen calc took 3 days and 4 hours. 

Original post:
Benzene has 21 occupied α and 21 occupied β orbitals.

How many core orbitals can we freeze when looking at electronic transitions, how does freezing core orbitals affect the energies, and what are the performance gains, if any?

I used the following relevant statements

tddft
    cis
    freeze core X
    nroots 24
end
task tddft energy

Also, xc b3lyp and 6-31++g**.

Freeze core 10 means freeze 10 α and freeze 10 β orbitals.

Results:

Frozen Runtime    Transitions with f>0
0          1111 s     6.7835, 6.8038, 6.8042, 7.3479, 7.3496
5          1107 s     6.7835, 6.8038, 6.8042, 7.3480, 7.3498
10        1063 s     6.7838, 6.8040, 6.8045, 7.3761, 7.3862
15        1063 s     6.5878, 6.7840, 6.8042, 6.8046
20          719 s     6.8038, 6.8355, 7.1692, 7.5334, 8.1866

Evaluation:
We're not really looking at what's right or wrong -- the main goal is to understand how the results are affected (we might accidentally get 'right' answer doing something stupid). Freezing more than 10 α/β orbitals leads to significant differences in predicted transitions.

Taking oscillation strength into account and plotting it, we see that we really don't want to overdo it with the number of frozen cores -- 15 and 20 frozen cores yield results that are about as predictable as a coin toss. We also don't see any overwhelming performance gains, but that may well be due to the size and computational cost (or lack thereof) of the system.

Octave code:

spec1=load('bz631gppdd_cosmo.dat');

spec2=load('bz631gppdd_cosmo_f5.dat');
spec3=load('bz631gppdd_cosmo_f10.dat');
spec4=load('bz631gppdd_cosmo_f15.dat');
spec5=load('bz631gppdd_cosmo_f20.dat');

gau= @(x,c,w,i) i*(1/(w*sqrt(2*pi))).*exp(-0.5.*((x-c)./w).^2);
x=linspace(160,200,200);
y1=cumsum(gau(x,1241.25./spec1(:,2),4,spec1(:,3)));
y2=cumsum(gau(x,1241.25./spec2(:,2),4,spec2(:,3)));
y3=cumsum(gau(x,1241.25./spec3(:,2),4,spec3(:,3)));
y4=cumsum(gau(x,1241.25./spec4(:,2),4,spec4(:,3)));
y5=cumsum(gau(x,1241.25./spec5(:,2),4,spec5(:,3)));


subplot(3,2,1)
 plot(x,y1(rows(y1),:))
 axis([160 200 0 0.2]);
 title('0 frozen');
 subplot(3,2,3)
 plot(x,y2(rows(y2),:))
 axis([160 200 0 0.2]);
 title('5 frozen');
 subplot(3,2,5)
 plot(x,y3(rows(y3),:))
 axis([160 200 0 0.2]);
 title('10 frozen');
 subplot(3,2,2)
 plot(x,y4(rows(y4),:))
 axis([160 200 0 0.2]);
 title('15 frozen');
 subplot(3,2,4)
 plot(x,y5(rows(y5),:))
 axis([160 200 0 0.2]);
 title('20 frozen');
 subplot(3,2,6)
  plot(x,y1(rows(y1),:),x,y2(rows(y2),:), x, y3(rows(y3),:), x,y4(rows(y4),:),x,y5(rows(y5),:))
  axis([160 200 0 0.2]);
 title('');