Ubuntu

January 21, 2011 Leave a comment

At the end of 2010, I decided it was time to buy a new laptop, specifically an Ubuntu laptop, to improve my Unix skills. I picked out a Pangolin laptop from System 76 with Ubuntu 10.10 “Maverick Meekrat” on it.

Moving from the Mac OS X and Windows XP paradigms to the Ubuntu paradigm is a bit jarring so far. My first annoyance has been installing the software I need on my new laptop. Ubuntu provides two GUIs for this, the Ubuntu Software Center and the Synaptic Package Manager. Synaptic is a developer-level GUI slapped onto the apt-get command line tool, and Ubuntu Software Center is clearly the snazzy way Canonical wants you to load new software.

But the stuff in their repositories is old! Sun Java is at 1.6.0_22 (latest is 23), Scala is at 2.7.7 (latest is 2.8.1), Eclipse is at 3.5.2 (latest is 3.6.1), git is at 1.7.1 (latest is 1.7.3.5). This bothers me. I want the latest stuff!

After a lot of searching, I discovered that some projects have accounts on launchpad.net. By adding the correct PPA URLs to the Other Repositories section of Software Sources, you could use both Synaptic and the Ubuntu Software Center to install the latest versions. Success!

Here are the PPA URLs I’ve found useful so far:

I don’t know why Scala, Eclipse, Netbeans and Sun Java don’t have Launchpad.net accounts to enable the easy package installation and updating features of Synaptic and USC.

My other bugaboo also applies to the USC. It’s clearly going for the same sort of functionality you find in Apple’s Mac App Store and iOS Store. But what I’m really missing is user feedback. For example, I did a search on GMail Notifiers and found multiple choices. Which one do I want? And I look at the Games section and there are A TON of games that sound cool, but which ones are essential? I don’t want to waste time installing and trying out game after game searching for the ones that are top notch. This should be integrated!

Advertisements
Categories: linux Tags:

Book Review: The Passionate Programmer by Chad Fowler

January 11, 2011 Leave a comment

The Passionate Programmer cover image

The Passionate Programmer is actually a second edition book; the first edition was entitled My Job Went To India (And All I Got Was This Lousy Book): 52 Ways To Save Your Job. Big improvement already!

The Passionate Programmer’s subtitle is Creating A Remarkable Career in Software Development. Through 53 short chapters, Chad Fowler inspires readers to do better and go further in our careers. Chad was a jazz saxophonist before making a change to software and his book illustrates many connections between the two fields. One point that stuck with me is practicing. Musicians obviously have to practice a lot before they get on stage and perform. Chad says developers should practice as well, before getting on the “stage” of delivering for a customer. He recommends doing your own studying, such as learning regular expressions or APIs that you are unfamiliar with, to increase your chops. Open source project work is an excellent way to gain experience implementing features, studying the code of other contributors of the project, and getting feedback by having your contribution reviewed.

The book is divided into five parts: Choosing Your Market; Investing In Your Product; Executing; Marketing… Not Just For Suits; and Maintaining Your Edge. Choosing Your Market focuses on planning your career by taking a high level view of the landscape and positioning yourself for maximum benefit. Investing In Your Product gives advice on how to improve your skills. Executing talks about the daily grind and how to make work more productive and enjoyable. Marketing is pretty self explanatory, giving some great advice on communicating how great you are to others. Maintaining Your Edge tells you how to avoid becoming a one hit wonder.

One topic that resonated with me was Chapter 36: Being Present. Chad talked about his time as CTO of a software development center in Bangalore, India and his troubles being so far removed geographically from the rest of the management team. His performance review mentioned a lack of “presence”, which is pretty obvious problem when you have such a hard time communicating with your managers. He relates this to coders’ sometimes antisocial tendencies to hole up in the cubicle and get into “the zone” of programming. This is an issue for us at Blackstone sometimes, because of the distributed nature of the projects and client sites. Maintaining that presence in the community of Blackstone is difficult but very important for both management perceptions and being able to leverage teamwork to achieve great things.

A few points in this book tied in tightly with the direction I feel Blackstone’s Company Contribution evaluation has been heading. Chapter 38: Change The World advises “Have a mission. Make sure people know it.” Chapter 42: Remarkability talks about doing things significantly different than those around you to be remarkable, such as releasing open source software, writing books and articles, and speaking at conferences. It’s not enough in this field to show up to work on time, write code on schedule and fix bugs quickly. Keeping up with the technology wave is important, but the ultimate goal should be to get out in front of it and show everyone a new way to go.

In the acknowledgments, Chad Fowler admits a huge debt to Dave Thomas and Andy Hunt’s great 1999 book The Pragmatic Programmer, which kicked off their entire book line. Both books contain a lot of great tips wrapped in short chapters that I will come back to frequently. I consider both to be essential and valuable, providing quick inspiration to improve both software and my career.

Categories: code Tags:

JAutodoc

January 5, 2011 1 comment

Things on my project have quieted down a bit, so I’m trying to get done some of the nice touches that I don’t usually have time to do. Because my module is brand new, it’s completely doable to make the inline documentation really spiffy. I want to do all the stuff to make the Javadoc look just right.

One of those things is to do package javadoc. You used to do that with a package.html file, but as of Java 5 they want you to create a package-info.java file instead. You can read all about it at http://www.oracle.com/technetwork/java/javase/documentation/index-137868.html

Because I use Eclipse, I figured there’d be some buried menu command to generate this stuff, but surprisingly there wasn’t. I did, however, find an Eclipse plugin called JAutodoc to do this for me.

Great! Except the plugin installation failed. The installation exception message mentioned a missing module, org.eclipse.team.cvs.ssh. Eclipse 3.6 Helios comes with the org.eclipse.team.cvs.ssh2 plugin, but for some reason they did not include org.eclipse.team.cvs.ssh.

I couldn’t find a site to download just that (54k) plugin jar file, but I did discover that Eclipse 3.5 Galileo did include it. So I moved that jar from my Galileo Eclipse installation to the Helios installation, retried the JAutodoc install, and everything works.

So if you want to be fussy about your Javadocs in Eclipse, check out JAutodoc, and if you have trouble installing it, this could be your solution!

Categories: code Tags: , ,

XOP Converter

December 18, 2010 2 comments

My most recent task at work involves XOP, or XML-binary Optimized Packaging. Basically, if you have an XML document that needs to have binary data in it, like images, you have two ways of doing it. You can base64 encode the data and put it directly in an XML element, or you can use XOP to keep the data in binary form and attach the file to the message using MIME. XOP is usually used to represent SOAP messages with MTOM (Message Transmission Optimization Mechanism).

My app uses JiBX to convert my Java objects into XML documents, and base64 encodes them. But I got a new requirement to also support XOP encoding. So to minimize impact on the app, I decided to write a converter that would take in a XML document with base64 encoded data and spit out a properly encoded XOP document. Then it would also be useful to convert existing documents and creating new ones. The module that helped me do this was Apache Axiom, a component of the Apache Axis2 web services engine.

The main problem with Axiom is its poor documentation. Almost all of the docs I could find about Axiom and Axis2 were from a server-side perspective, and I am writing a client-side app. So I thought I’d share the bit of code I wrote to do this conversion.

public class XOPConverter {

    @SuppressWarnings("rawtypes")
    public static void convertToXOP(String xml, OutputStream os) 
        throws XMLStreamException, JaxenException, JiBXException, IOException {

        // set up all the AXIOM objects
        XMLInputFactory factory = XMLInputFactory.newInstance();
        XMLStreamReader parser = factory.createXMLStreamReader(new StringReader(xml));
        StAXOMBuilder builder = new StAXOMBuilder(parser);
        OMDocument document = builder.getDocument();
        OMElement documentElement =  builder.getDocumentElement();
        OMFactory fac = OMAbstractFactory.getOMFactory();
        
        // find all the BinaryBase64Object nodes
        AXIOMXPath xpathExpression = new AXIOMXPath("//niem-nc:BinaryBase64Object");
        xpathExpression.addNamespace("niem-nc", "http://niem.gov/niem/niem-core/2.0");
        List nodeList = xpathExpression.selectNodes(documentElement);
        
        for (Iterator i = nodeList.iterator(); i.hasNext(); ) {
            Object obj = i.next();
            if (!(obj instanceof OMElement)) {
                continue;
            }
            OMElement element = (OMElement)obj;
            // get bytes data out of that node
            String base64data = element.getText();
            // decode base 64 encoding
            byte[] data = Base64Serializer.deserializeBase64(base64data);
            // create ByteArrayDataSource
            ByteArrayDataSource ds = new ByteArrayDataSource(data);
            // set the MIME type
            ds.setType("image/" + XOPConverter.getMIMEType(data));
            // create OMText object
            OMText textElement = fac.createOMText(new DataHandler(ds), true);
            // replace BinaryBase64Object node text with OMText object
            element.setText("");
            element.addChild(textElement);
        }
        
        // do the XOP conversion
        OMOutputFormat format = new OMOutputFormat();
        format.setDoOptimize(true);
        format.setMimeBoundary("MIME");
        format.setContentType("Multipart/Related");
        MTOMXMLStreamWriter mtomWriter = new MTOMXMLStreamWriter(os, format);
        mtomWriter.setDoOptimize(true);
        document.serializeAndConsume(mtomWriter);
        document.close(false);
    }
    
    /**
     * @param args
     * @throws IOException 
     * @throws XMLStreamException 
     * @throws JaxenException 
     * @throws JiBXException 
     */
    public static void main(String[] args) 
        throws IOException, XMLStreamException, JaxenException, JiBXException {
        
        if (args.length == 0) {
            System.out.println("Usage: <filename>");
            return;
        }
        String inputFileName = args[0];
        String outputFileName = inputFileName.substring(0, 
                inputFileName.lastIndexOf(".")) + ".xop";
        boolean overwrite = false;
        if (args.length > 1) {
            String overwriteStr = args[1];
            overwrite = Boolean.parseBoolean(overwriteStr);
        }
        // check if output file exists
        if (new java.io.File(outputFileName).exists() && !overwrite) {
            System.out.println("ERROR: Output file " + outputFileName + 
                    " exists, please move it.");
            return;
        }
        
        java.io.BufferedReader reader = 
            new java.io.BufferedReader(new java.io.FileReader(args[0]));
        String line = null;
        StringBuilder stringBuilder = new StringBuilder();
        String ls = System.getProperty("line.separator");
        while ((line = reader.readLine()) != null) {
            stringBuilder.append(line);
            stringBuilder.append(ls);
        }
        String xml = stringBuilder.toString();
        
        FileOutputStream os = new FileOutputStream(outputFileName);
        convertToXOP(xml, os);
        System.out.println("done");
    }
}

Now I’ll step through the code and explain it a bit.

The convertToXOP method takes the XML text as a String, and an OutputStream where the XOP is written. In my original document, all the base64 encoded objects I need to extract and attach are in elements labeled . So I cook up an XPath expression to find all those elements for processing. The code gets the data out of the element and un-base64 encodes it. I didn’t write that code, so you’ll have to do that yourself. Then we put the byte array in a ByteArrayDataSource and tag it with the MIME type we want to use. In a later post I’ll show how to figure out what encoding an image uses. Then use the OMFactory to create a new OMText element containing the byte array and set it to be optimized. Add it to the original element and reset its text, clearing out the base64 encoded data.

The key to doing the conversion is Axiom’s MTOMXMLStreamWriter class. You send the MTOMXMLStreamWriter the OutputStream where you want the XOP document to go, and an OMOutputFormat object. Setting the doOptimize flag on the OMOutputFormat object enables the optimization step that moves the binary objects to MIME attachments. Sending the MTOMXMLStreamWriter to the OMDocument’s serializeAndConsume method writes the XOP output.

I hope this code and explanation has been helpful to you.

Categories: code Tags: , ,

Links

December 12, 2010 Leave a comment

I wanted to share some links to some cool stuff I’ve found recently.

PragPub
This is a free monthly magazine by the folks to publish the Pragmatic Programmers series of books. They release the issues in pdf, epub and mobi formats in addition to HTML on the site. Lots of great stuff to load onto your ereader.

JAXenter
This site is a hub for software news. There’s a lot of press releases for Apache projects you’ve never heard of, but there’s also some Q&A with project managers and commentary on the goings on with Oracle and whatnot. Add the RSS feed to your feed reader and subscribe to their free occasional PDF magazine JAXmag.

The Java Posse
This is a group of four high-level developers who do a semi-regular podcast discussing Java. It can get really annoying when they talk over each other, and the episodes sometimes approach two rambling hours in length, but they are really smart and well-connected guys who are usually worth listening to.

Categories: miscellaneous Tags: , ,

DevIgnition

December 6, 2010 Leave a comment

Last Friday I took the day off of work to attend NOVAJUG’s free DevIgnition conference at Oracle HQ in Reston. It was a very interesting and informative series of presentations, and I’m definitely glad I set aside the time to check it out.

The first presenter was Suman Cuddapah, an Oracle SOA Solutions Specialist. He gave an overview of the recently released Java EE 6 updates and talked about the path forward for OpenJDK 7. His talk was pretty high level, obviously with no surprise revelations. The most interesting bit I took from his talk was the modularity introduced with Java EE 6, so you can deploy only the modules you need and keep the server stack lightweight.

The second talk was about Netbeans projects, presented by Ryan Cuprak. He did a quick survey of IDE usage in the crowd and the overwhelming favorite was Eclipse. I felt like his talk was intended to be evangelism for the Netbeans IDE, but it got mired down in an exhaustive list of the various project templates Netbeans provides. I think his talk would have been more effective if he had tried to highlight the advantages Netbeans has over Eclipse. It seemed like there were a lot of issues with Netbeans’s ability to import projects from Eclipse, further putting off potential switchers.

The third presenter was Reza Rahman, talking about how Spring 3 and Java EE interact. He went through a lot of Java EE technologies like JPA, EJB3, JMS and web services and showed how they can interoperate with Spring. If you are familiar with Spring and like some of the flexibility it gives you, you can leverage the best of both worlds.

The fourth presenter was Jay Garcia, talking about the Sencha Touch product. Sencha Touch is a JavaScript framework which came from ExtJS. It helps you build websites for mobile devices. The functionality Jay demoed was very slick. It allows you to code sites to function just like apps do on iOS, with HTML5, CSS and JavaScript. This is not an area I have done any work in before, but it looked quite snazzy.

The next presenters were David Bock and Arild Shirazi, talking about their effort to rewrite a legacy Swing and EJB app with JRuby and Rails. Their demo was kind of hard to follow, but the prospect of leaving Swing behind for something better was appealing.

The next talk was by self-described Clojure ninja Bryan Weber. After working with Scala for a couple of years, Bryan grew frustrated by the complexity and made the switch to Clojure, a variant of Lisp that runs on the JVM. It seems like there is a bit of a rivalry between Scala and Clojure to be the functional programming king on the JVM. I admit that I am someone who harbors deep reservations about Lisp programming. Bryan talked about the sometimes steep learning curve with Clojure, but his enthusiasm for the language was inspiring. These days I’m diving into learning Scala, but I’ll be keeping an ear out for Clojure and trying to see how the two compare.

The final presenter was Arun Gupta, talking about Java EE 6 and GlassFish. He knew that it’s tough for the last presenter of the day, so he really brought the energy to his talk. He had a huge slide deck but skipped most of it to focus on demoing code. He showed how Netbeans and GlassFish tightly integrate to speed development. I was highly pleased to see that Java EE 6 makes most of the XML configuration files optional, because I hate those cryptic things. Instead you can annotate your code. You can set up GlassFish as a deployment destination for your Netbeans project, and then every time you save a code change, it is automatically deployed.

I know a lot of very smart people are anti-IDE because it hides a lot of complexity from developers, and then they don’t really know what’s going on under the hood so they can’t fix problems. I was thinking about that position while I watched Arun Gupta fly through his very well prepared demo, inserting code templates and modifying existing code to demonstrate features. I am just a lot more productive in an IDE than I would be without it. I guess that IDEs can hide bad coders’ lack of abilities, but they also allow good coders to get more done faster. And they can make for some great demos!

All in all, it was an enlightening day and I’m already looking forward to the next conference!

Categories: socializing Tags:

Scala test drive

November 22, 2010 Leave a comment

Last week I got together with some coworkers to learn Scala and write some code. Since it was our first gathering, we kept it very simple and tried to really understand how the Scala code works and how it is different from Java.

Before the session, my day job gave me a very simple programming task, to read first and last names out of a directory of xml files and write them to a CSV file along with the name of the xml file it came from. There are many better and easier ways to do this, but I wrote it in Java, intending to rewrite it in Scala with my group.

Here’s the Java code:

import java.io.File;
import java.io.FileOutputStream;
import java.io.FileFilter;
import java.io.IOException;
import java.io.OutputStreamWriter;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathExpressionException;
import javax.xml.xpath.XPathFactory;
import org.w3c.dom.Document;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;

public class OTHExtractor {
    private static OutputStreamWriter writer = null;
    
    public static void main(String[] args) throws ParserConfigurationException,
            SAXException, IOException, XPathExpressionException {
        writer = new OutputStreamWriter(new FileOutputStream(new File("names.java.txt")));
        try {
            File[] files = new File(".").listFiles(new FileFilter() {
                public boolean accept(File file) { 
                    return file.getName().endsWith(".xml");
                } 
            });
            for (File file : files) {
                process(file);
            }
        } finally {
            writer.flush();
            writer.close();
        }
        System.out.println("done");
    }

    private static void process(File xmlFile)
            throws ParserConfigurationException, SAXException, IOException,
            XPathExpressionException {
        DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
        DocumentBuilder db = dbf.newDocumentBuilder();
        Document doc = db.parse(xmlFile);

        XPathFactory xpFactory = XPathFactory.newInstance();
        XPath xpath = xpFactory.newXPath();

        NodeList nodeList = (NodeList)xpath.evaluate("//PersonGivenName/text()", 
                doc, XPathConstants.NODESET);
        String givenName = "";
        if (nodeList != null && nodeList.getLength() > 0) {
            givenName = nodeList.item(0).getNodeValue();
        }

        nodeList = (NodeList)xpath.evaluate("//PersonSurName/text()", 
                doc, XPathConstants.NODESET);
        String surName = "";
        if (nodeList != null && nodeList.getLength() > 0) {
            surName = nodeList.item(0).getNodeValue();
        }
        writer.write(givenName + ", " + surName + ", " + xmlFile.getName() + "\n");
    }
}

And here’s the Scala code we wrote to replace it:

import java.io.PrintWriter
import scala.xml.XML
import java.io.File

object OTHExtractorScala {
  def main(args:Array[String]) = {
    val root = new File(".")
    
    val files = root.listFiles().filter(file => file.getName.endsWith(".xml"))
    val writer = new PrintWriter(new File("names.txt"))
    try {
      files foreach (file => writer.write(process(file)))
    } finally {
      writer.flush
      writer.close
    }
    println("done")
  }
  
  def process(file:File) = {
    val names = parseXML(file)
    names._1 + ", " + names._2 + ", " + names._3 + "\n"
  }
  
  def parseXML(file:File) = {
    val xml = XML.loadFile(file)
    ((xml\\"PersonGivenName").text, (xml\\"PersonSurName").text, file.getName)
  }
}

That’s 64 lines of verbose code whittled down to 29 very readable lines. Not bad!

Let’s take a closer look at some of this fancy Scala code.

    val files = root listFiles() filter(file => file.getName.endsWith(".xml"))

The first interesting thing about this line of code is that the call to filter the result of root’s listFiles call. We skipped the dots! You’re allowed to skip the dot and the parentheses when you call a function with zero or one arguments. Here we had to include the parentheses after listFiles so Scala would know that we’re calling a function called filter and not sending a value called filter to the listFiles function.

Next up is file => file.getName.endsWith(".xml")). What is this? This is an anonymous function. The bit before the => is the argument this function takes, in this case file. The part after the => is the body of the function. The Scala Array object’s filter method takes a function that returns Boolean. So this is a much slicker way of getting just the xml files than Java’s FileFilter interface.

The other thing I want to point out is the tuple. The parseXML method returns a tuple where the first element is the given name, the second element is the surname, and the third element is the file name. In Java we’d have to package them into a collection of some sort, or create an inner class to hold the values. Tuples make this much easier. The process method grabs the elements from the tuple names and turns them into a concatenated String separated by commas.

Oh, one other thing. Scala’s XML module ROCKS. Look how much easier it is to run XPath queries on documents. One line of code creates the XML document, one line of code finds the elements in the document. We needed a lot more verbose Java to do that.

Categories: code Tags: