Hadoop 0.20.1 - Null pointer exception on startup

I am trying out Hadoop on my Mac0SX (10.5.8). I unzipped the distribution and after cd'ing to ~/hadoop-0.20.1/bin, ran

ashee-mac:bin amitava$ ./start-all.sh

I get the following exception

2010-01-20 13:14:13,341 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = ashee-mac/10.17.8.21
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.20.1
STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1-rc1 -r 810220; compiled by 'oom' on Tue Sep  1 20:55:56 UTC 2009
************************************************************/
2010-01-20 13:14:13,480 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.lang.NullPointerException
 at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:134)
 at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:156)
 at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:160)
 at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:175)
 at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:279)
 at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
 at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)

2010-01-20 13:14:13,482 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************

---

Turns out, the default does not come configured with, well "defaults". The fix is easy enough as highlighted in Quick Start (duh)

conf/core-site.xml:

<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>

conf/hdfs-site.xml:

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>

conf/mapred-site.xml:

<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>

OpenOffice - BootstrapException: no office executable found!

I was working on building (via mvn) an open office based tool for extracting images from powerpoint. But upon invocation I was getting the following

[amitava@lamp target]$ ./run.sh --extract --input pptsample.ppt --output-dir /tmp/foo 
com.sun.star.comp.helper.BootstrapException: no office executable found!
 at com.sun.star.comp.helper.Bootstrap.bootstrap(Bootstrap.java:253)
 at edu.umich.med.umms.OpenOfficeUNODecomposition.main(OpenOfficeUNODecomposition.java:106)

Turns out, OO sdk wants you to reference the sdk jar's at the install location. I had to change the classpath like so

OO=/usr/lib64/openoffice.org/program/classes
CP=$CLASSPATH:./dcp-0.1.jar:$OO/juh.jar:$OO/jurt.jar:$OO/ridl.jar:$OO/unoil.jar
java -Djava.ext.dirs=./dependency -cp $CP edu.umich.med.umms.OpenOfficeUNODecomposition "$@"

This thread pointed me to the solution.

Data Visualization

http://www.juicekit.org/
http://vis.stanford.edu/protovis/
http://manyeyes.alphaworks.ibm.com/manyeyes/
http://flare.prefuse.org/

Apache Tika (Lucene) build issue - java.lang.OutOfMemoryError: Java heap space

While building apache tika from source, I get the following issue

$ mvn package

[INFO] [bundle:bundle {execution: default-bundle}]
java.lang.OutOfMemoryError: Java heap space
 at java.util.Arrays.copyOfRange(Arrays.java:3209)
 at java.lang.String.(String.java:215)
 at java.lang.StringBuilder.toString(StringBuilder.java:430)
 at aQute.lib.osgi.Clazz.normalize(Clazz.java:696)
 at aQute.lib.osgi.Clazz.doClassReference(Clazz.java:409)
 at aQute.lib.osgi.Clazz.parseClassFile(Clazz.java:262)
 at aQute.lib.osgi.Clazz.(Clazz.java:101)
 at aQute.lib.osgi.Analyzer.analyzeJar(Analyzer.java:1647)
 at aQute.lib.osgi.Analyzer.analyzeBundleClasspath(Analyzer.java:1563)
 at aQute.lib.osgi.Analyzer.analyze(Analyzer.java:108)
 at aQute.lib.osgi.Builder.analyze(Builder.java:294)
 at aQute.lib.osgi.Builder.doConditional(Builder.java:260)
 at aQute.lib.osgi.Builder.build(Builder.java:75)
 at org.apache.felix.bundleplugin.BundlePlugin.buildOSGiBundle(BundlePlugin.java:391)
 at org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:282)
 at org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:236)
 at org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:227)
 at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:483)
        ......
        ......

The solution is easy enough - just increase memory like so

$ MAVEN_OPTS="-Xmx3g" mvn package

Hope this helps folks who are getting tripped on this issue

ocamlfind - macosx

While exploring ocaml, the system could not find ocamlfind in my macport install of ocaml.

A quick google search lead me to this post

All I had to do is the following

amitava:learn amitava$ sudo port install caml-batteries 
--->  Computing dependencies for caml-batteries
--->  Fetching caml-findlib
--->  Attempting to fetch findlib-1.2.5.tar.gz from http://distfiles.macports.org/caml-findlib
--->  Verifying checksum(s) for caml-findlib
--->  Extracting caml-findlib
--->  Configuring caml-findlib
--->  Building caml-findlib
--->  Staging caml-findlib into destroot
--->  Installing caml-findlib @1.2.5_0
--->  Activating caml-findlib @1.2.5_0
--->  Cleaning caml-findlib
--->  Fetching caml-ounit
--->  Attempting to fetch ounit-1.0.3.tar.gz from http://distfiles.macports.org/caml-ounit
--->  Verifying checksum(s) for caml-ounit
--->  Extracting caml-ounit
--->  Configuring caml-ounit
--->  Building caml-ounit
--->  Staging caml-ounit into destroot
--->  Installing caml-ounit @1.0.3_0
--->  Activating caml-ounit @1.0.3_0
--->  Cleaning caml-ounit
--->  Fetching caml-type-conv
--->  Attempting to fetch type-conv-1.6.10.tar.gz from http://www.ocaml.info/ocaml_sources
--->  Verifying checksum(s) for caml-type-conv
--->  Extracting caml-type-conv
--->  Configuring caml-type-conv
--->  Building caml-type-conv
--->  Staging caml-type-conv into destroot
--->  Installing caml-type-conv @1.6.10_0
--->  Activating caml-type-conv @1.6.10_0
--->  Cleaning caml-type-conv
--->  Fetching caml-bin-prot
--->  Attempting to fetch bin-prot-release-1.2.20.tar.gz from http://www.janestreet.com/ocaml
--->  Verifying checksum(s) for caml-bin-prot
--->  Extracting caml-bin-prot
--->  Configuring caml-bin-prot
--->  Building caml-bin-prot
--->  Staging caml-bin-prot into destroot
--->  Installing caml-bin-prot @1.2.20_0
--->  Activating caml-bin-prot @1.2.20_0
--->  Cleaning caml-bin-prot
--->  Fetching caml-camlzip
--->  Attempting to fetch camlzip-1.04.tar.gz from http://distfiles.macports.org/caml-camlzip
--->  Verifying checksum(s) for caml-camlzip
--->  Extracting caml-camlzip
--->  Applying patches to caml-camlzip
--->  Configuring caml-camlzip
--->  Building caml-camlzip
--->  Staging caml-camlzip into destroot
--->  Installing caml-camlzip @1.04_0
--->  Activating caml-camlzip @1.04_0
--->  Cleaning caml-camlzip
--->  Fetching caml-camomile
--->  Attempting to fetch camomile-0.7.2.tar.bz2 from http://superb-east.dl.sourceforge.net/camomile
--->  Attempting to fetch camomile-0.7.2.tar.bz2 from http://superb-west.dl.sourceforge.net/camomile
--->  Attempting to fetch camomile-0.7.2.tar.bz2 from http://downloads.sourceforge.net/camomile
--->  Verifying checksum(s) for caml-camomile
--->  Extracting caml-camomile
--->  Configuring caml-camomile
--->  Building caml-camomile
--->  Staging caml-camomile into destroot
--->  Installing caml-camomile @0.7.2_0
--->  Activating caml-camomile @0.7.2_0
--->  Cleaning caml-camomile
--->  Fetching caml-pcre
--->  Attempting to fetch pcre-ocaml-6.0.1.tar.gz from http://www.ocaml.info/ocaml_sources/
--->  Verifying checksum(s) for caml-pcre
--->  Extracting caml-pcre
--->  Applying patches to caml-pcre
--->  Configuring caml-pcre
--->  Building caml-pcre
--->  Staging caml-pcre into destroot
--->  Installing caml-pcre @6.0.1_0
--->  Activating caml-pcre @6.0.1_0
--->  Cleaning caml-pcre
--->  Fetching caml-ocamlnet
--->  Attempting to fetch ocamlnet-2.2.9.tar.gz from http://distfiles.macports.org/caml-ocamlnet
--->  Verifying checksum(s) for caml-ocamlnet
--->  Extracting caml-ocamlnet
--->  Applying patches to caml-ocamlnet
--->  Configuring caml-ocamlnet
--->  Building caml-ocamlnet
--->  Staging caml-ocamlnet into destroot
--->  Installing caml-ocamlnet @2.2.9_0
--->  Activating caml-ocamlnet @2.2.9_0
--->  Cleaning caml-ocamlnet
--->  Fetching caml-sexplib
--->  Attempting to fetch release-4.2.16.tar.bz2 from http://distfiles.macports.org/caml-sexplib
--->  Verifying checksum(s) for caml-sexplib
--->  Extracting caml-sexplib
--->  Configuring caml-sexplib
--->  Building caml-sexplib
--->  Staging caml-sexplib into destroot
--->  Installing caml-sexplib @4.2.16_0
--->  Activating caml-sexplib @4.2.16_0
--->  Cleaning caml-sexplib
--->  Fetching caml-batteries
--->  Attempting to fetch batteries-20090405.tar.gz from http://distfiles.macports.org/caml-batteries
--->  Verifying checksum(s) for caml-batteries
--->  Extracting caml-batteries
--->  Configuring caml-batteries
--->  Building caml-batteries
--->  Staging caml-batteries into destroot
--->  Installing caml-batteries @20090405_0
--->  Activating caml-batteries @20090405_0
--->  Cleaning caml-batteries