Tailing Multiple Logs with Cygwin

August 14, 2013 1 comment

I just joined a new project at work building a workflow system using the Documentum successor Alfresco and its Business Process Management engine Activiti. When the project dev lead gave me a tour of developing for and using the system, I noticed that he opened Cygwin bash shells to tail three different Alfresco log files to look for errors. If you’re not of the software dev persuasion but are trying to make sense of this post for some reason, let me try to explain. Tail is a standard UNIX tool to show the end of a file from the command prompt, and using the -f option dynamically updates the display when new stuff gets added to the file. Thus tailing logs allows you to see the under-the-hood output from your software so you can figure out what the heck is really happening.

I realized that I would lose my mind tailing 3 different log files in 3 different Cygwin shells constantly, and immediately set out to make this process easier.

The first problem is that Alfresco makes new log files for each day, and uses the date in the filename. Thus the tail command needs the smarts to tail today’s log using the date. Here’s my bash script to tail Alfresco’s stdout log.

#!/bin/bash
today=`date +%Y-%m-%d`
tail --lines=200 -f /cygdrive/c/Alfresco/tomcat/logs/alfrescotomcat-stdout.$today.log

For this I want to run a single Windows batch file that kicks off 3 separate Cygwin shells for tailing the logs. That batch file is simple:

start /d "C:\cygwin64" Cygwin-tail-localhost.bat
start /d "C:\cygwin64" Cygwin-tail-stdout.bat
start /d "C:\cygwin64" Cygwin-tail-stderr.bat

Now comes the slightly tricky part, the batch files that kick off the Cygwin shells. Here’s what the standard Cygwin.bat script looks like:

@echo off
C:
chdir C:\cygwin64\bin
bash --login -i

I tried appending to the above bash command to run my ~/.tail-*.sh script, but no matter what I tried I could not customize the title of the window. This gave me 3 windows that looked the same and I couldn’t tell which window displayed which log file. So I abandoned the bash command and switched to Google’s mintty terminal emulator, and everything came together nicely. Here’s what Cygwin-tail-stdout.bat looks like:

@echo off
C:
chdir C:\cygwin64\bin
start mintty.exe -s 120,60 -t stdout /bin/bash -l -e './tail-stdout.sh'

Now I click on one icon in my Start menu and instantly 3 bash consoles appear tailing Alfresco’s logs. There are probably smarter ways to do this, but I’m pretty happy with this approach.

Categories: code Tags: , ,

Online Security Tips

May 15, 2013 Leave a comment

In my latest role as a developer focused on security, I thought I should round up some tips I’ve gathered on using the Internet more securely in 2013.

  • Use an web browser other than Internet Explorer. Google Chrome has been rated as the most secure browser, and has some nice auto-updating features.
  • Uninstall the Java browser plug-in. This is one of the most popular attack vectors for malware, and Oracle doesn’t make it very easy for users to keep Java up to date. You can also disable the browser plug-in functionality in the Java control panel if you prefer to keep Java installed. If you need Java for a specific website, consider installing the Java plug-in in a different browser, such as Safari or Firefox, that you use only for that website.
  • Adobe Flash and Adobe Reader are also prevalent attack vectors, though Adobe Reader has gotten a lot more secure with recent updates. Keep these programs up to date. Chrome keeps its version of Flash updated for you, another great reason to use it.
  • Use a free updating tool such as FileHippo to alert you to out of date and potentially vulnerable software on your computer.
  • On the second Tuesday of the month, Microsoft regularly releases critical software patches to Windows and other Microsoft software. If you are in charge of keeping your computer up to date with Windows Update, make sure to apply these patches ASAP. If your system administrator applies these patches, remember that these often require your computer to restart and plan accordingly. The window of time between the public release of these patches and your application of them is a period of heightened risk as attackers learn of these vulnerabilities and attempt to exploit them.
  • Password reuse is both a reality and a danger. We all have way too many passwords to remember, but reusing the same password on multiple sites increases the risk that one site will lose your password to criminals and compromise much more of your online identity. Use a secure password manager such as 1Password (stores passwords in a secure lockbox on your hard drive or Dropbox) or LastPass (stores passwords in its secure cloud service) to generate secure passwords for all your logins and populate them in your browser for you.
  • Many online services are now implementing 2 step authentication. This feature sends a code to your mobile device which is required to login from a previously unused browser or location. Services that support 2 step authentication include Yahoo, Google, Apple, Facebook and Dropbox, and the list is growing constantly. Enabling this can prevent your critical email, identity and cloud accounts from compromise by criminals.

Please consider following all of these steps to keep yourself safe and secure online.

Categories: security

Programming Fonts

October 26, 2011 1 comment

On the way in this morning I was listening to the most recent episode of the This Developer’s Life podcast which is all about fonts. I decided to spend a few minutes this morning sprucing up the fonts on my Windows XP desktop. The first thing I found was the ClearType Tuner PowerToy from Microsoft . Microsoft developed ClearType ages ago, but it was disabled by default on XP (enabled on Vista and 7). I turned that on and instantly everything looked better. Highly recommended if you still have XP.

Then I set about finding a better monospaced font in Eclipse than Courier. I found a number of suggestions for Consolas, a font Microsoft developed for Visual Studio. Download it if you don’t already have it. Other popular suggestions were DejaVu Mono and Andale Mono. Give them a try.

Categories: code Tags:

Patch Accepted

April 14, 2011 1 comment

Woo hoo! My patch to the maven-scala-plugin has been accepted!

That means that my GitHub pull request was accepted into the main repository. Users can get the fix by pulling down the code and building the plugin. The fix will be included in the next version of the plugin, which I will be sure to mention here!

Categories: code Tags: , ,

maven-scala-plugin

April 1, 2011 Leave a comment

Continuing from my last post: With the Scala script written, I wanted to integrate it into our Maven build as an optional step. But this didn’t go so well.

When running mvn scala:script with a link to my script file in the configured pom.xml, I got a Fatal Error. I also couldn’t get a simple helloWorld script to run, as a separate file or scripted right into the pom.xml file. This is the error I got:


[INFO] ------------------------------------------------------------------------
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] scala/ScalaObject
scala.ScalaObject
[INFO] ------------------------------------------------------------------------
[INFO] Trace
java.lang.NoClassDefFoundError: scala/ScalaObject
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org_scala_tools_maven.ScalaScriptMojo.runScript(ScalaScriptMojo.java:185)
at org_scala_tools_maven.ScalaScriptMojo.doExecute(ScalaScriptMojo.java:154)
at org_scala_tools_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:340)
at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: java.lang.ClassNotFoundException: scala.ScalaObject
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 33 more
[INFO] ------------------------------------------------------------------------

The maven-scala-plugin couldn’t even find the ScalaObject class, which is analagous to Java Object. Of course I googled the heck out of this error and found a few people suffering the same problem, but no one posted a solution. Then I found maven-scala-plugin’s repo on GitHub and discovered that this issue has been reported but not yet fixed. So I decided to download the source code and take a look.

Here’s what I found, in ScalaScriptMojo.java:

    private URLClassLoader createScriptClassloader(File scriptDir,
            Set<String> classpath) throws MalformedURLException {
        List<URL> urls = new ArrayList<URL>();

        // add the script directory to the classpath
        urls.add(scriptDir.toURI().toURL());

        for (String string : classpath) {
            urls.add(new URL("file://" + string));
        }

        URLClassLoader loader = new URLClassLoader(urls.toArray(new URL[urls
                .size()]), getClass().getClassLoader());
        return loader;
    }

That seemed like a pretty hinky way to me to send files to the URLClassLoader. The code that generates the Set of classpath Strings uses File.getCanonicalPath() to create the Strings. Thus they are platform-dependent Strings, and on a Windows box you can’t use them to create a valid File URL by prepending “file://” to them because the path separators are “\”. I assume that the plugin contributors are on Linux machines and couldn’t replicate the problem to fix it.

Like all the best bug fixes, this one was a one-liner:

for (String string : classpath) {
    urls.add(new File(string).toURI().toURL());
}

Now the URLs will be correct no matter what platform you’re on.

I forked the maven-scala-plugin repository on GitHub and checked in my fix. I was waiting to post this entry until my Pull Request was accepted by the repo owner, but I guess he is too busy working on other efforts. Until the request is accepted, please check out my forked repo.

Categories: code Tags: , ,

Scala script

March 31, 2011 Leave a comment

In my day job recently I found a way to sneak some Scala code into the project. I developed a datamodel module containing POJOs and JiBX bindings to marshal Objects into XML documents conforming to our XML schema. Some elements were constrained by enumerations, located either in the schema documents themselves or elsewhere in SQL dataload scripts. I stored these enumerations as properties files, so they could be easily edited when enumeration values changed.

Obviously this solution is not perfect. It violates the DRY (Don’t Repeat Yourself) principle, and enumeration value updates are likely to be forgotten. I decided to write a script that pulls the enumeration values out of their original locations and generates the properties files fresh, to ease maintainability. Scala includes the ability to run as a script, and its XML support is a lot better than Java’s, so I picked it.

Here’s an edited version of the script:

 import io.Source
 import java.io.{FileWriter, PrintWriter}
 import xml.{Node, Elem, XML}
 
 /**
  * This script will extract enumerations from schema and dataload files and write corresponding properties files.
  */
 
 val schemaDir = "C:\\Schemas\\ixm-ws-5.5\\xsd\\"
 val outputDir = "C:\\testharness\\datamodel\\src\\conf\\"
 
 // Get the enumerations out of jxdm.xsd
 var xsd = XML.loadFile(schemaDir + "subset\\jxdm\\3.0.3\\jxdm.xsd")
 
 extractEnumeration(xsd, "ActivityType.properties", "element", "ActivityTypeText", {node => (node \ "@value").text})
 extractEnumeration(xsd, "DocumentStatus.properties", "element", "StatusText", {node => (node \ "@value").text})
 
 // Get the enumerations out of ansi-nist.xsd
 xsd = XML.loadFile(schemaDir + "subset\\niem\\ansi-nist\\2.0\\ansi-nist.xsd")
 
 extractEnumeration(xsd, "ScaleUnits.properties", "simpleType", "SLCCodeSimpleType", {node => (node \\ "documentation").text})
 
 def extractEnumeration(xsd: Elem, propFileName: String, elementType: String, elementName: String, p: (Node) => String) {
   val out = new PrintWriter(outputDir + propFileName)
   xsd \\ elementType filter(node => (node \ "@name").text == elementName) foreach{(element) =>
     element \\ "enumeration" foreach{(enumeration) =>
       val value = (enumeration \ "@value").text
       val key = p(enumeration).trim.replace(" ", "\\ ")
       out.println(key + "=" + value)
     }
   }
   out.flush
   out.close
 }

The neatest part about this code is the last parameter to the extractEnumeration function: a function converting a Node to a String. Some of the enumeration values appear in nodes with the attribute “value” and some appear in nodes called “documentation”. I created closures that extracted the text out of each type of node and sent those functions as parameters to the extractEnumeration method. In Java I’d have to create a switch statement for each type of enumeration value extract and sent in a parameter to indicate which option to use. This way, when a new value extraction function is needed, we don’t have to touch the extractEnumeration function at all. I could have assigned the two closures to objects and reused them for all the extractEnumeration calls, but I thought this way made the code a little more readable.

My next idea for this was to integrate this script as an optional step in our maven build. But this didn’t go so well. Full details in the next post!

Categories: code Tags:

Book Review: Pragmatic Thinking & Learning

February 24, 2011 1 comment

Pragmatic Thinking & Learning Cover

Andy Hunt is best known as co-author of The Pragmatic Programmer and co-head of the publishing company spawned by that book. The mission of Andy’s 2008 book Pragmatic Thinking & Learning: Refactor Your Wetware is to get software professionals to engage their whole brain. I had heard about the division between left-brain and right-brain thinking, or conscious and subconscious thought, but Andy laid this division out in a new and very insightful way. He describes two modes of thinking: linear-mode, or L-mode and rich-mode, or R-mode. These two modes operate in very different ways. To unlock our full thinking power, we must learn to use the techniques Andy describes to quiet the chatter of our overused L-mode and let R-mode take over. As a novice yoga practitioner, the end-of-class meditation called savasana suddenly made sense not just as a relaxation technique, but also a way to engage R-mode thinking. Someone on Twitter recently posted “Meditation is a muscle”, and clearly this is an important one to exercise for knowledge workers.

The book describes a number of techniques for improving your learning, like mind maps and SQ3R (survey, question, read, recite, review). Andy also devotes a chapter to investigating your cognitive biases, including a very interesting discussion of the theory of Generational Affinity, or why the Boomers, Gen X and the Millennials think so differently. There’s also a much-needed section on managing your focus. Doing complex software work requires an immense amount of concentration, and distractions such as Outlook email notifications, Twitter feeds, unnecessary meetings or innocuous interruptions by co-workers can totally derail productivity. Especially in busy and distracting cubicle work environments, it’s important to find a way to achieve intense concentration, but it’s also important to find the time and environment to allow your R-mode a chance to work for you.

Pragmatic Thinking & Learning was a fantastic book. While I’m not sure Andy’s mind maps are really my thing, there was plenty in this book to absorb and use to further my career. I recently installed Evernote on all my computing devices to expand my exocortex and make sure I can capture ideas and thoughts wherever I am. This sort of deliberate thinking about thinking is what Pragmatic Thinking & Learning has to teach us, and I heartily recommend it even to non-technical people.

Categories: book review Tags: