Paul Vodra's Blog

In my day job recently I found a way to sneak some Scala code into the project. I developed a datamodel module containing POJOs and JiBX bindings to marshal Objects into XML documents conforming to our XML schema. Some elements were constrained by enumerations, located either in the schema documents themselves or elsewhere in SQL dataload scripts. I stored these enumerations as properties files, so they could be easily edited when enumeration values changed.

Obviously this solution is not perfect. It violates the DRY (Don’t Repeat Yourself) principle, and enumeration value updates are likely to be forgotten. I decided to write a script that pulls the enumeration values out of their original locations and generates the properties files fresh, to ease maintainability. Scala includes the ability to run as a script, and its XML support is a lot better than Java’s, so I picked it.

Here’s an edited version of the script:

 import io.Source
 import java.io.{FileWriter, PrintWriter}
 import xml.{Node, Elem, XML}
 
 /**
  * This script will extract enumerations from schema and dataload files and write corresponding properties files.
  */
 
 val schemaDir = "C:\\Schemas\\ixm-ws-5.5\\xsd\\"
 val outputDir = "C:\\testharness\\datamodel\\src\\conf\\"
 
 // Get the enumerations out of jxdm.xsd
 var xsd = XML.loadFile(schemaDir + "subset\\jxdm\\3.0.3\\jxdm.xsd")
 
 extractEnumeration(xsd, "ActivityType.properties", "element", "ActivityTypeText", {node => (node \ "@value").text})
 extractEnumeration(xsd, "DocumentStatus.properties", "element", "StatusText", {node => (node \ "@value").text})
 
 // Get the enumerations out of ansi-nist.xsd
 xsd = XML.loadFile(schemaDir + "subset\\niem\\ansi-nist\\2.0\\ansi-nist.xsd")
 
 extractEnumeration(xsd, "ScaleUnits.properties", "simpleType", "SLCCodeSimpleType", {node => (node \\ "documentation").text})
 
 def extractEnumeration(xsd: Elem, propFileName: String, elementType: String, elementName: String, p: (Node) => String) {
   val out = new PrintWriter(outputDir + propFileName)
   xsd \\ elementType filter(node => (node \ "@name").text == elementName) foreach{(element) =>
     element \\ "enumeration" foreach{(enumeration) =>
       val value = (enumeration \ "@value").text
       val key = p(enumeration).trim.replace(" ", "\\ ")
       out.println(key + "=" + value)
     }
   }
   out.flush
   out.close
 }

The neatest part about this code is the last parameter to the extractEnumeration function: a function converting a Node to a String. Some of the enumeration values appear in nodes with the attribute “value” and some appear in nodes called “documentation”. I created closures that extracted the text out of each type of node and sent those functions as parameters to the extractEnumeration method. In Java I’d have to create a switch statement for each type of enumeration value extract and sent in a parameter to indicate which option to use. This way, when a new value extraction function is needed, we don’t have to touch the extractEnumeration function at all. I could have assigned the two closures to objects and reused them for all the extractEnumeration calls, but I thought this way made the code a little more readable.

My next idea for this was to integrate this script as an optional step in our maven build. But this didn’t go so well. Full details in the next post!

Last week I got together with some coworkers to learn Scala and write some code. Since it was our first gathering, we kept it very simple and tried to really understand how the Scala code works and how it is different from Java.

Before the session, my day job gave me a very simple programming task, to read first and last names out of a directory of xml files and write them to a CSV file along with the name of the xml file it came from. There are many better and easier ways to do this, but I wrote it in Java, intending to rewrite it in Scala with my group.

Here’s the Java code:

import java.io.File;
import java.io.FileOutputStream;
import java.io.FileFilter;
import java.io.IOException;
import java.io.OutputStreamWriter;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathExpressionException;
import javax.xml.xpath.XPathFactory;
import org.w3c.dom.Document;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;

public class OTHExtractor {
    private static OutputStreamWriter writer = null;
    
    public static void main(String[] args) throws ParserConfigurationException,
            SAXException, IOException, XPathExpressionException {
        writer = new OutputStreamWriter(new FileOutputStream(new File("names.java.txt")));
        try {
            File[] files = new File(".").listFiles(new FileFilter() {
                public boolean accept(File file) { 
                    return file.getName().endsWith(".xml");
                } 
            });
            for (File file : files) {
                process(file);
            }
        } finally {
            writer.flush();
            writer.close();
        }
        System.out.println("done");
    }

    private static void process(File xmlFile)
            throws ParserConfigurationException, SAXException, IOException,
            XPathExpressionException {
        DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
        DocumentBuilder db = dbf.newDocumentBuilder();
        Document doc = db.parse(xmlFile);

        XPathFactory xpFactory = XPathFactory.newInstance();
        XPath xpath = xpFactory.newXPath();

        NodeList nodeList = (NodeList)xpath.evaluate("//PersonGivenName/text()", 
                doc, XPathConstants.NODESET);
        String givenName = "";
        if (nodeList != null && nodeList.getLength() > 0) {
            givenName = nodeList.item(0).getNodeValue();
        }

        nodeList = (NodeList)xpath.evaluate("//PersonSurName/text()", 
                doc, XPathConstants.NODESET);
        String surName = "";
        if (nodeList != null && nodeList.getLength() > 0) {
            surName = nodeList.item(0).getNodeValue();
        }
        writer.write(givenName + ", " + surName + ", " + xmlFile.getName() + "\n");
    }
}

And here’s the Scala code we wrote to replace it:

import java.io.PrintWriter
import scala.xml.XML
import java.io.File

object OTHExtractorScala {
  def main(args:Array[String]) = {
    val root = new File(".")
    
    val files = root.listFiles().filter(file => file.getName.endsWith(".xml"))
    val writer = new PrintWriter(new File("names.txt"))
    try {
      files foreach (file => writer.write(process(file)))
    } finally {
      writer.flush
      writer.close
    }
    println("done")
  }
  
  def process(file:File) = {
    val names = parseXML(file)
    names._1 + ", " + names._2 + ", " + names._3 + "\n"
  }
  
  def parseXML(file:File) = {
    val xml = XML.loadFile(file)
    ((xml\\"PersonGivenName").text, (xml\\"PersonSurName").text, file.getName)
  }
}

That’s 64 lines of verbose code whittled down to 29 very readable lines. Not bad!

Let’s take a closer look at some of this fancy Scala code.

    val files = root listFiles() filter(file => file.getName.endsWith(".xml"))

The first interesting thing about this line of code is that the call to filter the result of root’s listFiles call. We skipped the dots! You’re allowed to skip the dot and the parentheses when you call a function with zero or one arguments. Here we had to include the parentheses after listFiles so Scala would know that we’re calling a function called filter and not sending a value called filter to the listFiles function.

Next up is file => file.getName.endsWith(".xml")). What is this? This is an anonymous function. The bit before the => is the argument this function takes, in this case file. The part after the => is the body of the function. The Scala Array object’s filter method takes a function that returns Boolean. So this is a much slicker way of getting just the xml files than Java’s FileFilter interface.

The other thing I want to point out is the tuple. The parseXML method returns a tuple where the first element is the given name, the second element is the surname, and the third element is the file name. In Java we’d have to package them into a collection of some sort, or create an inner class to hold the values. Tuples make this much easier. The process method grabs the elements from the tuple names and turns them into a concatenated String separated by commas.

Oh, one other thing. Scala’s XML module ROCKS. Look how much easier it is to run XPath queries on documents. One line of code creates the XML document, one line of code finds the elements in the document. We needed a lot more verbose Java to do that.