Paul Vodra's Blog

In my day job recently I found a way to sneak some Scala code into the project. I developed a datamodel module containing POJOs and JiBX bindings to marshal Objects into XML documents conforming to our XML schema. Some elements were constrained by enumerations, located either in the schema documents themselves or elsewhere in SQL dataload scripts. I stored these enumerations as properties files, so they could be easily edited when enumeration values changed.

Obviously this solution is not perfect. It violates the DRY (Don’t Repeat Yourself) principle, and enumeration value updates are likely to be forgotten. I decided to write a script that pulls the enumeration values out of their original locations and generates the properties files fresh, to ease maintainability. Scala includes the ability to run as a script, and its XML support is a lot better than Java’s, so I picked it.

Here’s an edited version of the script:

 import io.Source
 import java.io.{FileWriter, PrintWriter}
 import xml.{Node, Elem, XML}
 
 /**
  * This script will extract enumerations from schema and dataload files and write corresponding properties files.
  */
 
 val schemaDir = "C:\\Schemas\\ixm-ws-5.5\\xsd\\"
 val outputDir = "C:\\testharness\\datamodel\\src\\conf\\"
 
 // Get the enumerations out of jxdm.xsd
 var xsd = XML.loadFile(schemaDir + "subset\\jxdm\\3.0.3\\jxdm.xsd")
 
 extractEnumeration(xsd, "ActivityType.properties", "element", "ActivityTypeText", {node => (node \ "@value").text})
 extractEnumeration(xsd, "DocumentStatus.properties", "element", "StatusText", {node => (node \ "@value").text})
 
 // Get the enumerations out of ansi-nist.xsd
 xsd = XML.loadFile(schemaDir + "subset\\niem\\ansi-nist\\2.0\\ansi-nist.xsd")
 
 extractEnumeration(xsd, "ScaleUnits.properties", "simpleType", "SLCCodeSimpleType", {node => (node \\ "documentation").text})
 
 def extractEnumeration(xsd: Elem, propFileName: String, elementType: String, elementName: String, p: (Node) => String) {
   val out = new PrintWriter(outputDir + propFileName)
   xsd \\ elementType filter(node => (node \ "@name").text == elementName) foreach{(element) =>
     element \\ "enumeration" foreach{(enumeration) =>
       val value = (enumeration \ "@value").text
       val key = p(enumeration).trim.replace(" ", "\\ ")
       out.println(key + "=" + value)
     }
   }
   out.flush
   out.close
 }

The neatest part about this code is the last parameter to the extractEnumeration function: a function converting a Node to a String. Some of the enumeration values appear in nodes with the attribute “value” and some appear in nodes called “documentation”. I created closures that extracted the text out of each type of node and sent those functions as parameters to the extractEnumeration method. In Java I’d have to create a switch statement for each type of enumeration value extract and sent in a parameter to indicate which option to use. This way, when a new value extraction function is needed, we don’t have to touch the extractEnumeration function at all. I could have assigned the two closures to objects and reused them for all the extractEnumeration calls, but I thought this way made the code a little more readable.

My next idea for this was to integrate this script as an optional step in our maven build. But this didn’t go so well. Full details in the next post!

Last week I got together with some coworkers to learn Scala and write some code. Since it was our first gathering, we kept it very simple and tried to really understand how the Scala code works and how it is different from Java.

Before the session, my day job gave me a very simple programming task, to read first and last names out of a directory of xml files and write them to a CSV file along with the name of the xml file it came from. There are many better and easier ways to do this, but I wrote it in Java, intending to rewrite it in Scala with my group.

Here’s the Java code:

import java.io.File;
import java.io.FileOutputStream;
import java.io.FileFilter;
import java.io.IOException;
import java.io.OutputStreamWriter;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathExpressionException;
import javax.xml.xpath.XPathFactory;
import org.w3c.dom.Document;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;

public class OTHExtractor {
    private static OutputStreamWriter writer = null;
    
    public static void main(String[] args) throws ParserConfigurationException,
            SAXException, IOException, XPathExpressionException {
        writer = new OutputStreamWriter(new FileOutputStream(new File("names.java.txt")));
        try {
            File[] files = new File(".").listFiles(new FileFilter() {
                public boolean accept(File file) { 
                    return file.getName().endsWith(".xml");
                } 
            });
            for (File file : files) {
                process(file);
            }
        } finally {
            writer.flush();
            writer.close();
        }
        System.out.println("done");
    }

    private static void process(File xmlFile)
            throws ParserConfigurationException, SAXException, IOException,
            XPathExpressionException {
        DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
        DocumentBuilder db = dbf.newDocumentBuilder();
        Document doc = db.parse(xmlFile);

        XPathFactory xpFactory = XPathFactory.newInstance();
        XPath xpath = xpFactory.newXPath();

        NodeList nodeList = (NodeList)xpath.evaluate("//PersonGivenName/text()", 
                doc, XPathConstants.NODESET);
        String givenName = "";
        if (nodeList != null && nodeList.getLength() > 0) {
            givenName = nodeList.item(0).getNodeValue();
        }

        nodeList = (NodeList)xpath.evaluate("//PersonSurName/text()", 
                doc, XPathConstants.NODESET);
        String surName = "";
        if (nodeList != null && nodeList.getLength() > 0) {
            surName = nodeList.item(0).getNodeValue();
        }
        writer.write(givenName + ", " + surName + ", " + xmlFile.getName() + "\n");
    }
}

And here’s the Scala code we wrote to replace it:

import java.io.PrintWriter
import scala.xml.XML
import java.io.File

object OTHExtractorScala {
  def main(args:Array[String]) = {
    val root = new File(".")
    
    val files = root.listFiles().filter(file => file.getName.endsWith(".xml"))
    val writer = new PrintWriter(new File("names.txt"))
    try {
      files foreach (file => writer.write(process(file)))
    } finally {
      writer.flush
      writer.close
    }
    println("done")
  }
  
  def process(file:File) = {
    val names = parseXML(file)
    names._1 + ", " + names._2 + ", " + names._3 + "\n"
  }
  
  def parseXML(file:File) = {
    val xml = XML.loadFile(file)
    ((xml\\"PersonGivenName").text, (xml\\"PersonSurName").text, file.getName)
  }
}

That’s 64 lines of verbose code whittled down to 29 very readable lines. Not bad!

Let’s take a closer look at some of this fancy Scala code.

    val files = root listFiles() filter(file => file.getName.endsWith(".xml"))

The first interesting thing about this line of code is that the call to filter the result of root’s listFiles call. We skipped the dots! You’re allowed to skip the dot and the parentheses when you call a function with zero or one arguments. Here we had to include the parentheses after listFiles so Scala would know that we’re calling a function called filter and not sending a value called filter to the listFiles function.

Next up is file => file.getName.endsWith(".xml")). What is this? This is an anonymous function. The bit before the => is the argument this function takes, in this case file. The part after the => is the body of the function. The Scala Array object’s filter method takes a function that returns Boolean. So this is a much slicker way of getting just the xml files than Java’s FileFilter interface.

The other thing I want to point out is the tuple. The parseXML method returns a tuple where the first element is the given name, the second element is the surname, and the third element is the file name. In Java we’d have to package them into a collection of some sort, or create an inner class to hold the values. Tuples make this much easier. The process method grabs the elements from the tuple names and turns them into a concatenated String separated by commas.

Oh, one other thing. Scala’s XML module ROCKS. Look how much easier it is to run XPath queries on documents. One line of code creates the XML document, one line of code finds the elements in the document. We needed a lot more verbose Java to do that.

What is Scala?

In their book The Pragmatic Programmer, Andrew Hunt and David Thomas suggest “Learn a new language every year.” I looked around at the hot new languages out there today and decided that Scala was the most promising one to learn.

Scala, short for Scalable Language, is both a functional language and an object-oriented language. Like Groovy and JRuby, Scala runs on the Java Virtual Machine, which means it’s easy to integrate Scala code with existing Java code. It is a statically typed langauge, like Java, which means its type checking is performed during compile-time, as opposed to dynamically typed languages like Groovy and Ruby whose type checking is performed at runtime.

Scala code can be compiled and run as class files, like Java, or you can run it as a script directly from the .scala file. There’s even an interactive command line mode for quickly trying out some code.

Scala has a very concise syntax. It tries to reduce the signal-to-noise ratio of Java code, making the code more readable. It makes a lot of the dots and parentheses of Java code optional, which can be disconcerting for old school Java heads. Clearly the mark of an experienced Scala developer will be the conciseness of their code.

Scala supports a lot of nifty language features like tuples, closures and currying, which makes it a good language to learn what the heck those are and how they might be useful to you in your coding. In general, Scala feels like Java 2.0 in the same way that Java felt like C++ 2.0.

Quick History of Scala

Scala was originally released in late 2003 by its creator, Martin Odersky. The most recent version of Scala is 2.8.0, released in August 2010.

A free web framework called Lift supports Scala code in an environment very similar to Ruby on Rails.

A number of high profile websites have recently switched from Ruby on Rails to Scala & Lift, including Twitter and Foursquare.

James Strachan, the original creator of Groovy, famously said in a blog post in 2009 that if someone had shown him the book Programming In Scala, by Scala’s creator Martin Odersky, he probably wouldn’t have created Groovy.

Programming Scala Book Review

One of Scala’s big draws is its Actor model for concurrent programming. The book I read, Programming Scala by Venkat Subramaniam, showed its focus on this area in its subtitle, “Tackle Multicore Complexity on the Java Virtual Machine”. These days, the new microchips that power our computers feature more and more cores on a single chip. To take advantage of that multicore power, software must be written to take advantage of concurrent processing.

Anyone who’s worked with Java threads knows how painful they can be to get right. Scala’s emphasis on functional programming, and the Actor model, make writing concurrent code a lot easier than in Java. Functional programming avoids state and mutable data, making it ideal for concurrency. The Actor model helps you dispatch functions to perform work and gather the results when each task completes. Anyone needing to write highly scalable code should check out Scala for these valuable benefits.

The rest of Subramaniam’s book provides an excellent overview of Scala. Progamming Scala is part of The Pragmatic Programmer series of books, which strive to be highly readable and concise. At only 208 pages, Programming Scala gets through a lot of material but doesn’t go very deep into any of it. I found myself struggling to grasp the complexity of Scala’s syntax and wished that Subramaniam had gone into more detail in that area.

I’d recommend Programming Scala as a good introduction to the language, but one would probably need to supplement it with a few web based tutorials. I am looking forward to checking out Martin Odersky’s Programming In Scala book soon, which I hope to teach me a lot more about how to write good Scala code.