112. Kernel 3.3.x on debian testing

Compiling a kernel on debian is easy. Kernel 3.3 came out today, and here's how to build it for debian testing:
sudo apt-get install kernel-package fakeroot
wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.3.tar.bz2
tar -xvf linux-3.3.tar.bz2 
cd linux-3.3/

cat /boot/config-`uname -r`>.config
make oldconfig

EDIT: 3.3.1 is here: http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.3.1.tar.bz2 -- the build instructions are the same
EDIT 2: This has been tried with 3.3.4 as well. All is fine.
EDIT 3: And it's fine with 3.3.5

You'll also be asked about the new inclusions in the kernel. You can pick the default if you don't know - Yes means to compile into the kernel, m mean to provide as a module and no means don't support. Some drivers are better provided as modules -- see e.g. http://justlinux.com/forum/archive/index.php/t-127876.html
"Some things obviously have to be compiled into the kernel - file system support for your / filesys, stuff like that. Most everything else can be modules, if you desire." and "f you hot-swap different usb peripherals then I would recommend compiling those drivers as modules". Same goes for network drivers.

You can also look up the different options here: http://cateee.net/lkddb/web-lkddb/.
Seems like there's a lot of network and touch/screen drivers. And the fabled android drivers are available now:

Make audit loginuid immutable (AUDIT_LOGINUID_IMMUTABLE) [N/y/?] (NEW) Yes
Memory Resource Controller Kernel Memory accounting (EXPERIMENTAL) (CGROUP_MEM_RES_CTLR_KMEM) [N/y/?] (NEW) No
EFI stub support (EFI_STUB) [N/y/?] (NEW) No
UNIX: socket monitoring interface (UNIX_DIAG) [N/m/y/?] (NEW) m
UDP: socket monitoring interface (INET_UDP_DIAG) [N/m/?] (NEW)  m
Netfilter NFACCT over NFNETLINK interface (NETFILTER_NETLINK_ACCT) [N/m/y/?] (NEW) m
Supply CT list in procfs (OBSOLETE) (NF_CONNTRACK_PROCFS) [Y/n/?] (NEW) Yes
"nfacct" match support (NETFILTER_XT_MATCH_NFACCT) [N/m/?] (NEW) No
IPVS source hashing table size (the Nth power of 2) (IP_VS_SH_TAB_BITS) [8] (NEW)
"rpfilter" reverse path filter match support (IP_NF_MATCH_RPFILTER) [N/m/?] (NEW) No
"rpfilter" reverse path filter match support (IP6_NF_MATCH_RPFILTER) [N/m/?] (NEW) No
Open vSwitch (OPENVSWITCH) [N/m/y/?] (NEW) No
Network priority cgroup (NETPRIO_CGROUP) [N/m/y/?] (NEW) m
ISA Bus based legacy SJA1000 driver (CAN_SJA1000_ISA) [N/m/?] (NEW) No
Bosch CC770 and Intel AN82527 devices (CAN_CC770) [N/m] (NEW) No
NFC LLCP support (EXPERIMENTAL) (NFC_LLCP) [N/y/?] (NEW) No
Block Device Driver for Micron PCIe SSDs (BLK_DEV_PCIESSD_MTIP32XX) [N/m/?] (NEW) No
NVM Express block device (BLK_DEV_NVME) [N/m/y/?] (NEW) No
Ethernet team driver support (EXPERIMENTAL) (NET_TEAM) [N/m/y/?] (NEW) Yes
Calxeda 1G/10G XGMAC Ethernet driver (NET_CALXEDA_XGMAC) [N/m/y/?] (NEW) No
Micrel KS8995MA 5-ports 10/100 managed Ethernet switch (MICREL_KS8995MA) [N/m/y] (NEW) No
Atheros ath9k bluetooth coexistence support (ATH9K_BTCOEX_SUPPORT) [Y/n/?] (NEW) Yes
Hardware support that overlaps with the brcmsmac driver (B43_BCMA_EXTRA) [Y/n] (NEW) Yes
Broadcom IEEE802.11n PCIe SoftMAC WLAN driver (BRCMSMAC) [N/m/?] (NEW) No
iwlwifi experimental P2P support (IWLWIFI_P2P) [N/y/?] (NEW) No
Enable full debugging output in iwlegacy (iwl 3945/4965) drivers (IWLEGACY_DEBUG) [N/y/?] (NEW)  No
TCA8418 Keypad Support (KEYBOARD_TCA8418) [N/m/?] (NEW) No
AUO in-cell touchscreen using Pixcir ICs (TOUCHSCREEN_AUO_PIXCIR) [N/m/?] (NEW) No
EETI eGalax multi-touch panel support (TOUCHSCREEN_EGALAX) [N/m/?] (NEW) No
PIXCIR I2C touchscreens (TOUCHSCREEN_PIXCIR) [N/m/?] (NEW) No
Sharp GP2AP002A00F I2C Proximity/Opto sensor driver (INPUT_GP2A) [N/m/?] (NEW) No
Polled GPIO tilt switch (INPUT_GPIO_TILT_POLLED) [N/m/y/?] (NEW) No
SBS Compliant gas gauge (BATTERY_SBS) [N/m/?] (NEW) No
National Semiconductor LP8727 charger driver (CHARGER_LP8727) [N/m/?] (NEW) No
Battery charger manager for multiple chargers (CHARGER_MANAGER) [N/y/?] (NEW)  No?
VIA Watchdog Timer (VIA_WDT) [N/m/y/?] (NEW) No
Support STMicroelectronics STMPE (MFD_STMPE) [N/y/?] (NEW) No
Support Dialog Semiconductor DA9052/53 PMIC variants with SPI (MFD_DA9052_SPI) [N/y/?] (NEW) No
Enable IR raw decoder for the Sanyo protocol (IR_SANYO_DECODER) [M/n/?] (NEW) No
JL2005B/C/D USB V4L2 driver (USB_GSPCA_JL2005BCD) [N/m/?] (NEW) m
V4L PCI(e) devices (V4L_PCI_DRIVERS) [Y/n/?] (NEW) Yes
V4L ISA and parallel port devices (V4L_ISA_PARPORT_DRIVERS) [N/y/?] (NEW) No
V4L platform devices (V4L_PLATFORM_DRIVERS) [N/y/?] (NEW)  No
Intel GMA5/600 KMS Framebuffer (DRM_GMA500) [N/m/?] (NEW) No
Roccat Isku keyboard support (HID_ROCCAT_ISKU) [N/m/?] (NEW) No
Microsoft Hyper-V mouse driver (HID_HYPERV_MOUSE) [N/m/?] (NEW) No
EHCI support for Marvell on-chip controller (USB_EHCI_MV) [N/y/?] (NEW) No
Inventra Highspeed Dual Role Controller (TI, ADI, ...) (USB_MUSB_HDRC) [N/m/?] (NEW) No
Marvell USB2.0 Device Controller (USB_MV_UDC) [N/m/?] (NEW) No
LED Support for TCA6507 I2C chip (LEDS_TCA6507) [N/m/?] (NEW) No
LED support for the Bachmann OT200 (LEDS_OT200) [N/m/y/?] (NEW) No
InfiniBand SCSI RDMA Protocol target support (INFINIBAND_SRPT) [N/m/?] (NEW) No
Support for rtllib wireless devices (RTLLIB) [N/m/?] (NEW)
Android Drivers (ANDROID) [N/y/?] (NEW) No
Fujitsu Tablet Extras (FUJITSU_TABLET) [N/m/y/?] (NEW) No
Fujitsu-Siemens Amilo rfkill support (AMILO_RFKILL) [N/m/?] (NEW) No
AMD IOMMU Version 2 driver (EXPERIMENTAL) (AMD_IOMMU_V2) [N/m/y/?] (NEW) No
Btrfs with integrity check tool compiled in (DANGEROUS) (BTRFS_FS_CHECK_INTEGRITY) [N/y/?] (NEW) No
NFS server manual fault injection (NFSD_FAULT_INJECTION) [N/y/?] (NEW) No
Kernel memory leak detector (DEBUG_KMEMLEAK) [N/y/?] (NEW) No
NMI Selftest (DEBUG_NMI_SELFTEST) [N/y/?] (NEW) No
Serpent cipher algorithm (x86_64/SSE2) (CRYPTO_SERPENT_SSE2_X86_64) [N/m/y/?] (NEW) 
No

Continuing,
make-kpkg clean
fakeroot make-kpkg -j7 --initrd --revision=3.3.0 --append-to-version=amd64 kernel_image kernel_headers  

The build takes a LONG time. Once it's done:

mv ../linux*3.3.0*.deb .
sudo dpkg -i *.deb

Done.

Linux tantalum 3.3.0-amd64 #1 SMP Tue Mar 20 06:29:46 EST 2012 x86_64 GNU/Linux

---

111. Ecce (nwchem) on Debian, and ROCKS/Centos

If you're using nwchem chances are that you've considered using ECCE to parse the output:
http://ecce.emsl.pnl.gov/

First of all you'll need to register at https://eus.emsl.pnl.gov/Portal/ -- and you can only do that if you're faculty. Postdocs and PhD students need not apply. Other than that, it's free, but you'll have to wait a couple of days to get your registration approved.

As much as I like nwchem owing to the clear syntax, I feel less warmly about ecce. Don't get me wrong -- it's pretty. It's just feels archaic and cobbled together. Even worse is that it's not open source and that its workings feel a bit opaque at times. Still, there's no better program for visually parsing nwchem output at this point. Anyway...

--start here --
Debian:
Download the install_ecce.v6.2.rhel5-gcc3.2.3-m32.csh file to ~/tmp/ecce

There's no md5sum supplied but here's what I got:
2ee70cc817dee9f80b11be5eac6e53e5

If you haven't already
sudo apt-get install csh 

OK, moving on...
cd ~/tmp/ecce
chmod +x  install_ecce.v6.2.rhel5-gcc3.2.3-m32.csh
./install_ecce.v6.2.rhel5-gcc3.2.3-m32.csh

Main ECCE installation menu
===========================
0) Help on main menu options
1) Full install
2) Full upgrade
3) Application software install
4) Application software upgrade
5) Server install
6) Server upgrade

Pick 1 if you're installing on your desktop and there's no server that you know of. 

Once the installation is over you get:

***************************************************************
!! You MUST perform the following steps in order to use ECCE !!
-- Unless only the user 'me' will be running ECCE,
   start the ECCE server as 'me' with:
     /home/me/tmp/ecce/ecce-v6.2/server/ecce-utils/start_ecce_server
-- To register machines to run computational codes, please see
   the installation and compute resource registration manuals
   at http://ecce.pnl.gov/using/installguide.shtml
-- To run ECCE each user must source either the runtime_setup
   (csh/tcsh) or runtime_setup.sh (sh/bash/ksh) script in the
   directory /home/me/tmp/ecce/ecce-v6.2/apps/scripts
   from their shell environment setup script.  For example,
   with csh or tcsh, add the following to ~/.cshrc:
     if (-e /home/me/tmp/ecce/ecce-v6.2/apps/scripts/runtime_setup) then
       source /home/me/tmp/ecce/ecce-v6.2/apps/scripts/runtime_setup
     endif
***************************************************************

Which translates to:
1. sh  /home/me/tmp/ecce/ecce-v6.2/server/ecce-utils/start_ecce_server
2. Sourcing that file makes no sense. Instead, add the following to your ~/.bashrc

export ECCE_HOME=/home/me/tmp/ecce/ecce-v6.2/apps

export PATH=${ECCE_HOME}/scripts:${PATH}

Assuming you've source your ~/.bashrc, start ecce by typing
ecce

...which takes an unreasonably long time (ca 1 min) after which you're greeted by

Press Any Key

Type in a password -- any password -- which will be your password from now on.
You're then taken to

Click on Viewer (assuming you've got something to look at)

Pay attention to the fine print

Have a look at the text box in the bottom right corner..and pay attention. In my particular case I have 6 cores and an mpi aware nwchem 6.0 version compiled. I bet that's better than whatever comes bundled with ecce. Also, the

To change you go to the machine browser (see screen shot #2), click on set up remote access and make sure that everything is working by clicking on e.g. processes:

Then click on the Machine menu (top left), select Register Machine while your machine is selected.

You can now change your options.

Running:
So, before using ecce you always need to
sh  /home/me/tmp/ecce/ecce-v6.2/server/ecce-utils/start_ecce_server
first. The server will run until you stop it or reboot.
Next, start ecce
ecce

Integration with nwchem
Most people would probably set up their nwchem jobs by hand, because it's so simple. All you need to do is to include the statement
ecce_print ecce.out
in the beginning, and you'll get an ecce.out file which you can then IMPORT (not open regularly, but import) into ecce.

Click on Viewer, Import Calculation From Output File, select your ecce out and voilá:

ECCE: homo (benzene)

If you're running debian, you're done now.

---

---

ROCKS 5.4.3/Centos 5.6:
This isn't a fix as much as a rant. The problem with ROCKS 5.4.3 is that csh is so broken that it's a struggle just to install ecce. I mean, I do show how to get ecce running in the end, but ROCKS feels like an unfinished piece of work compared to a normal debian install.

--Demonstration only -- don't do --
First back up ssh-key.sh and ssh-key.csh in /etc/profile.d

So...you start by
chmod +x install_ecce.v6.2.rhel5-gcc3.2.3-m32.csh
./install_ecce.v6.2.rhel5-gcc3.2.3-m32.csh
...and nothing's happening.

You then try just typing in
csh

/etc/profile.d/ssh-key.sh: line 211: return: can only `return' from a function or sourced script
It appears that you have not set up your ssh key.
This process will make the files:
     /export/home/me/.ssh/id_rsa.pub
     /export/home/me/.ssh/id_rsa
     /export/home/me/.ssh/authorized_keys
Generating public/private rsa key pair.
/export/home/me/.ssh/id_rsa already exists.
Overwrite (y/n)? 

Turns out there's a bug in ROCKS 5.4.3.  You can fix that by:
rpm -Uvh ftp://www.rocksclusters.org/pub/rocks/updates/5.4.3/x86_64/RPMS/rocks-config-server-5.4.3-1.x86_64.rpm

So far so good.
csh
...and nothing. It just exits. Or so you think. But the problem is bigger than that --  try opening a new terminal in e.g. gnome (gnome-terminal or xterm) -- it exits immediately. No error message or anything.

You can get csh to start by moving /etc/csh.cshrc out of the way, but you're still screwed as to opening a new terminal. The only way to get back a working system is to restore ssh-key.sh and ssh-key.csh.

--- Demonstration over ---

--Start here --
 You could also get around all this by running
csh -f
But then you don't have any env. variables loading and it can lead to problems of its own.

Anyway:
csh -f install_ecce.v6.2.rhel5-gcc3.2.3-m32.csh

The install starts. Just follow the instructions.

After installation, start the server:
csh -f ecce-v6.2/server/ecce-utils/start_ecce_server

Hit enter until you get a workable prompt back...
Edit your ~/.bashrc and add

export ECCE_HOME=/home/me/tmp/ecce/ecce-v6.2/apps

export PATH=${ECCE_HOME}/scripts:${PATH}

Don't bother sourcing your ~/.bashrc. It's easier to just open a new terminal.
Type
ecce
and you should be up and running...sort of. Under ROCKS I had problems importing ecce.out files since I had problems actually connecting to the server. Don't know why, but it came down to not being able to open a remote shell on the host.

NOTE:
this worked fine on one box, but not on another one which I was setting up remotely. On that one I had to edit

ecce/apps/siteconfig/Dataservers

and

ecce/apps/siteconfig/jndi.properties 

In particular, I had to change references to eccetera.emsl.pnl.gov.

110. Compiling, installing Gnuplot 4.6 on Debian

A new version of gnuplot doesn't happen very often, and this one has an interesting added feature in terms of support for using braces in algorithms.
http://www.gnuplot.info/announce.4.6.0

Building gnuplot 4.6 is similar to building 4.4.4 and is pretty straightforward:

sudo apt-get install libgd2-xpm-dev checkinstall

wget http://sourceforge.net/projects/gnuplot/files/latest/download?source=files
mv download\?source\=files gnuplot-4.6.tar.gz
tar -xvf gnuplot-4.6.tar.gz
cd gnuplot-4.6.0/

./configure --with-linux-vga
make
checkinstall -install=no
 sudo rm /usr/local/share/info/dir -rf
sudo dpkg -i gnuplot_4.6.0-1_amd64.deb

You may get an error if trying to install on a system with a home-compiled version of octave (see below).

The problem with handling small numbers is not present in this version (http://verahill.blogspot.com.au/2012/02/debian-testing-wheezy-64-bug-in-debian.html).

---

Error:

Selecting previously unselected package gnuplot.
(Reading database ... 258722 files and directories currently installed.)
Unpacking gnuplot (from gnuplot_4.6.0-1_amd64.deb) ...
dpkg: error processing gnuplot_4.6.0-1_amd64.deb (--install):
 trying to overwrite '/usr/local/share/info/dir', which is also in package octave 3.6.1-1
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Errors were encountered while processing:
 gnuplot_4.6.0-1_amd64.deb

Solution:

 sudo dpkg --force-overwrite -i gnuplot_4.6.0-1_amd64.deb