Last week I got Sakai automatically building on the NMI test facility at Wisconsin. Here is a status page:
http://grandcentral.cs.wisc.edu/pavlo/index.php?page=results%2Foverview&opt_user=csev
Here are my scripts in SVN:
https://source.sakaiproject.org/contrib/csev/trunk/nmi/sakaimvn/
The submit file is the master control - here is my sample:
$ cat sakaiMaven2.submit
project = sakai
component = Trunk Build
component_version = 1.0.0
description = Auto-build of Sakai trunk (cron)
run_type = Build
inputs = sakaiMaven2.scp, sakaiMaven2.svn, m2-repo.scp
remote_task = buildSakai.sh
remote_task_args = "sakai/trunk sakai"
platforms = x86_macos_10.4
prereqs_x86_macos_10.4 = java-1.5.0_06
notify = csev@umich.edu
I need Java 1.5 - the simplest is to use the mac OS 10.4 systems for my builds. Later I will develop scripts for other operating systems.
The inputs specify what you want on your execution machine and how to get it there. I have three inputs.
The first copies all my shell scripts from the current directory to the destination machine using SCP.
[csev@nmi-s003 sakaimvn]$ cat sakaiMaven2.scp
method = scp
scp_file = /home/csev/sakaimvn/*.sh
recursive = true
The second input checks a particular tag out of SVN of the Sakai source.
[csev@nmi-s003 sakaimvn]$ cat sakaiMaven2.svn
method = svn
url = https://source.sakaiproject.org/svn/sakai/trunk sakai
The third copies a copy of a populated Maven 2 repository from my home directory on the submitting machine to the executing machine.
[csev@nmi-s003 sakaimvn]$ cat m2-repo.scp
method = scp
scp_file = /home/csev/.m2
recursive = true
As the source changes the .m2 repo will become out of date so I have to tar up a full repo from my dev machine from time to time and copy it over.
The script to be run is pretty complex - I will point out a few important steps.
In link-repo.sh the key line is
ln -s `pwd`/.m2 ~/.m2
The input directive will insure that the .m2 directory makes it into the execution directory - by soft linking it into home, Maven 2 will find it.
In nmi-bootstrap.sh the code is a bit tortuous - probably the clever bit is downloading a copy of Maven 2 and using the local copy.
I debugged this script separately on my mac before making it work on the NMI test bed.
I instlal Maven 2 each time the script runs - I use:
curl -O http://mirrors.ibiblio.org/pub/mirrors/apache/maven/binaries/maven-2.0.7-bin.tar.gz
Then extract it into my local directory and simply use
`pwd`/maven-2.0.7/bin/mvn
As my call to maven 2. This way I install my own pre-requisite and don't have to depend on anything except for Java.
Once this is done, I put in a crontab to automate this:
[csev@nmi-s003 ~]$ crontab -l
MAILTO=csev@umich.edu
22 04 * * * $HOME/nightly.sh
[csev@nmi-s003 ~]$ cat nightly.sh
#!/bin/sh
PATH=$PATH:/nmi/bin:/usr/local/condor/bin
cd sakaimvn
nmi_submit sakaiMaven2.submit
Thanks to Jim Basney and everyone who helped walk me through the process. it took less than a day to get it all working.
Also thanks to the folks at Wisconsin who walked me through the process.
One of the things I do in by launch script is dump out variables and check the version of Java I have just for my sanity. Here is that part of the script:
which javac
javac -version
echo
echo shell vars
set
echo Env vars
env
Here is that output:
Which java are we running
/prereq/java-1.5.0_06/bin/java
Where is curl
/usr/bin/curl
What about javac
/usr/bin/javac
shell vars
BASH=/bin/sh
BASH_VERSINFO=([0]="2" [1]="05b" [2]="0" [3]="1" [4]="release" [5]="powerpc-apple-darwin8.0")
BASH_VERSION='2.05b.0(1)-release'
DIRSTACK=()
EUID=4294967294
GROUPS=()
HOME=/Users/condor/execute/dir_3996
HOSTNAME=nmi-0068.cs.wisc.edu
HOSTTYPE=powerpc
IFS='
'
MACHTYPE=powerpc-apple-darwin8.0
NMI_BIN=/Users/condor/execute/dir_3996/bin
NMI_CONDOR_RELEASE_DIR=NMI_PLACEHOLDER_DEFAULT_VALUE
NMI_GNUTAR=tar
NMI_PLATFORM=x86_macos_10.4
NMI_USER_HAWKEYE_DIR=/tmp/nmi-hawkeye/
NMI_component=SakaielloWorld
NMI_component_version=1.0.0
NMI_description='This is a simple example'
NMI_inputs=sakaiMaven2.scp
NMI_notify='
NMI_platforms=x86_macos_10.4
NMI_prereqs=java-1.5.0_06
NMI_project=sakai
NMI_remote_task=helloWorld.sh
NMI_remote_task_args='"Remote_Task Task"'
NMI_run_type=test
OPTERR=1
OPTIND=1
OSTYPE=darwin8.0
PATH=/prereq/java-1.5.0_06/bin:/bin:/usr/bin:/Users/condor/execute/dir_3996/userdir
PIPESTATUS=([0]="0")
POSIXLY_CORRECT=y
PPID=3998
PS4='+ '
PWD=/Users/condor/execute/dir_3996/userdir
SHELL=/usr/bin/false
SHELLOPTS=braceexpand:hashall:interactive-comments:posix
SHLVL=2
TERM=dumb
UID=4294967294
USER=nobody
_=vars
_CONDOR_ANCESTOR_171=8042:1186004412:1008764864
_CONDOR_ANCESTOR_3996=3998:1186671661:3296594176
_CONDOR_ANCESTOR_8042=3996:1186671660:1921037768
_CONDOR_SCRATCH_DIR=/Users/condor/execute/dir_3996
_CONDOR_SLOT=1
_NMI_PREREQ_java_1_5_0_06_ROOT=/prereq/java-1.5.0_06
_NMI_TASKNAME=remote_task
Env vars
NMI_USER_HAWKEYE_DIR=/tmp/nmi-hawkeye/
_NMI_TASKNAME=remote_task
NMI_inputs=sakaiMaven2.scp
NMI_description=This is a simple example
_CONDOR_SCRATCH_DIR=/Users/condor/execute/dir_3996
NMI_project=sakai
USER=nobody
NMI_component=SakaielloWorld
_CONDOR_ANCESTOR_8042=3996:1186671660:1921037768
NMI_notify=
PATH=/prereq/java-1.5.0_06/bin:/bin:/usr/bin:/Users/condor/execute/dir_3996/userdir
NMI_GNUTAR=tar
PWD=/Users/condor/execute/dir_3996/userdir
NMI_PLATFORM=x86_macos_10.4
_NMI_PREREQ_java_1_5_0_06_ROOT=/prereq/java-1.5.0_06
NMI_remote_task_args="Remote_Task Task"
_CONDOR_SLOT=1
SHLVL=2
HOME=/Users/condor/execute/dir_3996
NMI_remote_task=helloWorld.sh
NMI_component_version=1.0.0
_CONDOR_ANCESTOR_3996=3998:1186671661:3296594176
NMI_platforms=x86_macos_10.4
NMI_CONDOR_RELEASE_DIR=NMI_PLACEHOLDER_DEFAULT_VALUE
NMI_prereqs=java-1.5.0_06
NMI_run_type=test
_CONDOR_ANCESTOR_171=8042:1186004412:1008764864
NMI_BIN=/Users/condor/execute/dir_3996/bin
_=/usr/bin/env