Build a Talking Calendar with GoogleCL and Android Scripting Environment

At the end of my post on how to Setup GoogleCL on WinXP I joked about running GoogleCL, Google's new command line tool for accessing Google service, on the Android OS. Trying to make that joke a reality, I found the Android Scripting Environment project which allows running Python scripts on Android - step one: accomplished.

Warning: this is not for the faint of heart. The instructions that follow will require you to not only install software on your Android device, but also to download and modify Python source code. I will assume that you are capable of using SVN to get source code from code.google.com and applying a .patch file. And you'll probably end up having to type on a really tiny keyboard at some point. Don't say I didn't warn you.

Install Android Scripting Environment (ASE)

The easiest way to install ASE is to open up the ASE home page on your PC and scan the barcode.

Hack GoogleCL

Note: this change is now in trunk and won't be necessary in the 0.9.8 release.

Now grab the source code for GoogleCL (see here).

I needed to apply a small patch to get my setup to work with the 0.9.7 release.

The Patch

Index: src/googlecl/__init__.py
===================================================================
--- src/googlecl/__init__.py (revision 299)
+++ src/googlecl/__init__.py (working copy)
@@ -156,7 +156,7 @@
   import pickle
   token_path = os.path.join(GOOGLE_CL_DIR, TOKENS_FILENAME_FORMAT % user)
   if os.path.exists(token_path):
-    with open(token_path, 'r') as token_file:
+    with open(token_path, 'rb') as token_file:
       token_dict = pickle.load(token_file)
     try:
       token = token_dict[service.lower()]
@@ -184,7 +184,7 @@
   token_path = os.path.join(GOOGLE_CL_DIR, TOKENS_FILENAME_FORMAT % user)
   success = False
   if os.path.exists(token_path):
-    with open(token_path, 'r+') as token_file:
+    with open(token_path, 'rb+') as token_file:
       token_dict = pickle.load(token_file)
       try:
         del token_dict[service.lower()]
@@ -236,12 +236,12 @@
   import stat
   token_path = os.path.join(GOOGLE_CL_DIR, TOKENS_FILENAME_FORMAT % user)
   if os.path.exists(token_path):
-    with open(token_path, 'r') as token_file:
+    with open(token_path, 'rb') as token_file:
       token_dict = pickle.load(token_file)
   else:
     token_dict = {}
   token_dict[service] = token 
-  with open(token_path, 'w') as token_file:
+  with open(token_path, 'wb') as token_file:
     # Ensure only the owner of the file has read/write permission
     os.chmod(token_path, stat.S_IRUSR | stat.S_IWUSR)
     pickle.dump(token_dict, token_file)

I made it small on purpose because it's hard to read anyway. Hopefully, this way you'll be able to select it all and copy it easily. Apply the patch to your GoogleCL code.

The patch makes GoogleCL open the auth token file in binary mode. I needed this because later I'll be copying a token file from my desktop to my phone. Without the patch, the phone rejected token the file.

Install GoogleCL

Now you need to copy your GoogleCL source onto the phone. Your Android phone may vary. On mine, I just connect to my desktop computer via USB and select "Disk Drive" as the USB connection type. This mounts the phone's storage as two removable disks. You can also remove the SD card from your phone and put it in a reader connected to your computer, since it's the card we're putting files on.

Find ase/extras/python/ on your SD card. This directory should contain a number of Python modules already, such as gdata and xml. Copy your googlecl/src/googlecl in next to them. Also copy googlecl/src/google to ase/scripts/google.py. You'll use this later when you write your talking calendar scripts.

Write a Script to Call GoogleCL

This script changes your HOME dir to /sdcard so that GoogleCL can write your .googlecl configuration file. I wasn't able to figure out how to let GoogleCL write to the default HOME location. The other settings are what ASE uses when it runs Python scripts. I don't know how many of them are required to actually make this work, and I haven't felt like fiddling with it too much since it's working.

Replace [USERNAME] with your Google account username and save it as ase/scripts/calendar_today.sh.

#!/bin/bash 
export HOME=/sdcard 
export TEMP="/sdcard/ase/extras/pythontmp" 
export PYTHONHOME="/data/data/com.google.ase/python" 
export PYTHONPATH="/sdcard/ase/extras/python:/sdcard/ase/scripts/"
/data/data/com.google.ase/python/bin/python /sdcard/ase/scripts/google.py 
    --user [USERNAME] calendar list --date `date +%Y-%m-%d`

Make It Talk

I love this part because it's so easy with Android. This script calls the one above and passes the output to the phone's text-to-speech service. Save it as ase/scripts/say_calendar.py.

import android 
import subprocess  
p = subprocess.Popen(
        ['/system/bin/sh',  '/sdcard/ase/scripts/calendar_today.sh'],
        stdout=subprocess.PIPE, 
        stderr=subprocess.PIPE)
output, errors = p.communicate()
print errors 
print output 
droid = android.Android() 
droid.ttsSpeak(output)

Important Note for Windows Users: be sure to convert any files written on Windows to UNIX line endings before copying them to your phone. If you need to, you can also open up the files on your phone and remove the extra line end characters there. ASE has a file editor built in and I use ASTRO for editing other files.

Preauthentication

I couldn't get the GoogleCL OAuth process to work on my phone. When I returned to ASE from completing the auth process in the browser the script had already stopped running, interrupting the authentication. I got around that by preparing a auth token file on my desktop and copying it to the phone.

Using your modified copy of GoogleCL on the desktop, authenticate to each of the services you want to use from your phone. Then take the access_tok_[username] file from the .googlecl directory under your home directory and put that file in /sdcard/.googlecl/ on your phone.

Dismount your phone's drives from your desktop, or reinstall the SD card in your phone.

Make It So

Now, assuming you've followed my instructions exactly and I haven't left out anything important, you should be able to:

• Open up ASE on your phone
• Select Menu->View->Interpreters
• Select Menu->Add->Python
• Press back to return to the list of scripts

You should see a number of scripts listed, including the ones you just wrote. Select say_calendar.py. Your calendar entries for today should be displayed and the phone should speak them soon after. Now, it may not sound the way you expect - the TTS isn't perfect. But I think that's pretty good for a few minutes setting up some free scripting tools.

I would like to extend a very big thank you to Jason Holt and Tom Miller. Their encouragement and advice made this possible. Thanks, guys.

Richmond JUG April 21st

I went to my second Java User Group meeting last night. Kevin Smith gave a talk on Service-oriented Architecture (SOA) security. He talked about using interceptors and about access control based on services, requests, or individual records (or even individual pieces of data), and about loose coupling. And that's really what it all comes down to: loose coupling. All the buzz, hype, and mania over SOA really just boils down to the simple practice of not tying all of your code up in knots. Write little pieces of code that do one thing and anywhere one piece depends on another piece, use a layer of abstraction (an interface in the Object-oriented world or a service in SOA-speak) so that component A isn't tied to the implementation of component B, but rather to a specification that could be satisfied by components C or D instead. When the components don't depend on each other directly, it's more likely you'll be able to reuse one or more of them later.

When Peer Reviews Attack

I'm a big fan of peer review in software development. I love sharing ideas, getting feedback on my work, and finding ways to improve my code. But in over 7 years as a developer, I've never seen reviews be done as often as they should. And, sadly, several that I have seen and participated in have gone horribly wrong. The failure usually boils down to poor communication. A bad review meeting, or chain of emails, can degenerate into finger pointing, hurt feelings, and ultimately failure to achieve the goal of the review process: high quality code.

Why review?

Peer review brings many benefits to a software project. It's an opportunity to have another pair (or more) of eyes on a developer's work, trying to catch subtle (or simply careless) mistakes, suggest ways to improve code, or ensure adherence to standards. Peer review is collaborative; the reviewer is taking a stake in the original work and will be backing it with their own seal of approval at the end of the review process. Problems will be caught sooner, standards will be followed consistently, and both reviewer and reviewee will have a chance to learn from the collaboration and perform better in the future.

What could go wrong?

Plenty. For example:

• Expectations for the process aren't set up front. A developer who doesn't know what to expect from the process may not be prepared to participate properly.
• Standards aren't communicated in advance. Peer review shouldn't play out like a game of Mao.
• The reviewer is overzealous. Peer review isn't a time to pursue one's own programming ideaology. That's what the standards committee was for!
• The reviewee takes it personally. Remember that it isn't about you or your worth as a developer. It's about working together to produce high quality code.

I've seen each of these dramas. They lead to alienation of team members, reduced productivity, and sometimes the abandonment of the entire review process. But I've also seen code reviews that work. Good reviews allow ideas to flow freely and the result is ultimately a better solution. Even if I'm only sitting on the sidelines of a review, I can learn something from the process that makes me a better developer.

What makes a better review process?

• Document the process. Make sure developers are aware of the review process before they begin any work that will be reviewed. If possible, have a new developer sit in on a review first. Reviewers and reviewees should know how the process will work, what they're expected to bring to the process, and what the expected results are.
• Document the standards. Make sure you have clear, reasonable, and up-to-date standards documentation available to the reviewee before they begin any work that will be reviewed.
• Be specific. The reviewer should point out not only which rules were violated, but also where. Give the file name, line number, and a brief description of the violation.
• Be flexible. The reviewer should ask questions and suggest alternatives, but keep in mind that there's a difference between a violation or bug and an option he or she might prefer (if you can't prove empirically that it's better, then it isn't a blocker). The reviewee should do his or her best to follow standards and document code where necessary, be ready to fix problems that are found, and be open to considering suggestions.

It's a jury of your peers

I think this one point is worth stressing. In the best reviews I've seen done, everyone is treated as an equal. It isn't a senior developer lecturing a junior on proper technique, it's two (or more) developers who may not be of equal experience or skill, but nevertheless regard each other, and the process, with respect.

GWT, and AppEngine, and Eclipse - oh my!

There's much to be happy about for Java web developers this morning. Google has announced support for Java on AppEngine. Early access is being granted on a FIFO basis, however you can download the SDK now. But wait, there's more! This SDK includes the long-awaited Eclipse plugin for GWT! Read all about it on the GWT blog.

That's made my day.

NullPointerException

Why write about NullPointerExceptions?

NullPointerExceptions are one of the most basic runtime errors encountered in Java programming. Learning how to fix NullPointerExceptions is a vital skill for anyone programming in Java, as any student or amateur Java programmer will encounter these errors eventually. Sadly, even some professional Java programmers are stumped by them. I hope that this article will help programmers understand what NullPointerExceptions are, why they happen, and how to fix them. And for when all else fails, I'll offer a few pointers on asking for assistance with a NullPointerException.

What is a NullPointerException?

A NullPointerException is an unchecked runtime error. As such, NullPointerExceptions do not have to be declared in a method's throws clause. Because they are not checked exceptions, it is easy to forget about them and many programmers will not make any effort to handle or avoid NullPointerExceptions until they start occurring. A NullPointerException may indicate a logical error, improper data validation, incorrect use of an API, or some other programming problem.

Why do NullPointerExceptions happen?

A NullPointerException is caused by an attempt to dereference a pointer that doesn't point to anything; the pointer is null. Here are a couple of common scenarios where NullPointerExceptions can be found:

1.    String myString = null;
2.    System.out.println(myString.length());

Here a variable, myString, is declared and initialized to null on line #1. When line #2 attempts to deference the variable myString in order to print the string's length, a NullPointerException is thrown because myString doesn't point to anything.

1.    System.out.println(aMethodThatReturnsNull().toString());

This second example is a little trickier. No variable was declared, but there is still a pointer: the return value of the method aMethodThatReturnsNull(). That fictitious (and seemingly useless) method will always return null. As we saw in the first example, attempting to dereference a null pointer and call a method on the referenced object (which is null) results in a NullPointerException.

How do you track down a NullPointerException?

To find the source of a NullPointerException, start with the stack trace. In your Java console or log file, you'll see something like this:

Exception in thread "main" java.lang.NullPointerException
	at com.foo.example.NullPointerExample.main(NullPointerExample.java:21)

The stack trace tells you what happened (NullPointerException) and where (line 21 of NullPointerExample.java). Look at that line and see if you can recognize one of the two patterns given above. Ask yourself these questions:

1. Are you calling a method on a variable that might be null?
2. Are you calling a method on the return value of another method, where the first method might return null?
3. Are you absolutely sure the answer to the first two questions was "no?" NullPointerExceptions can happen for other reasons, but these two are by far the most common.

How do you ask for help with a NullPointerException?

First, put some effort into it. Although most developer communities are ready and willing to help with problems relating to the community's subject area, NullPointerExceptions are very often simple programming errors and probably off topic for anything except a Java beginners forum. So before posting and asking for help, determine if the problem is in your own code.

Once you're certain that the exception isn't arising from your code, take a look at the code that is causing the exception (see above). You might be using an API improperly. Third-party libraries may not support null parameters, for example.

If you still need help, be sure to follow these guidelines when posting on a forum or emailing colleagues directly:

• Be polite. You're asking for other people's time, generally without paying for it.
• Be thorough.
  • Include the stack trace from the exception.
  • Include any of your code that is relevant to the stack trace.
  • Explain what you've done to troubleshoot and detail your findings.
• Be patient. People are busy and have their own priorities. It may take hours or days to get a reply from the community, and longer to get a free solution. If you need immediate, devoted attention, you should expect to pay for it. If you want to "bump" your message to get more attention, don't just post "bump" or "can anyone help" -- offer up some additional information to demonstrate that you're taking ownership of the problem and still working on it (nobody wants to do your job for you!). Your new findings may elicit a response. Whining about not getting a response is unlikely to help matters and can alienate the community members who could assist you.

GWT 1.6 Milestone 1 Announced

Scott Blum from the GWT team announced the release of GWT 1.6 Milestone 1 on the GWT Contributor's forum yesteday evening. 1.6 will include, among other things, many bug fixes, support for a new project structure that more closely resembles a standard WAR file, and a new event system that originated in the GWT Incubator.

GWT in Eclipse

I came across this video last week. It's a presentation by Bruce Johnson, Technical Lead for the Google Web Toolkit on developing GWT applications with the Eclipse IDE. The first part of the presentation covers some basic introduction to GWT, followed by a little more detail on the inner workings of JSNI and hosted mode. The part I found most interesting was Bruce's discussion of the Eclipse plugin for GWT that Google has been working on internally but has not, as of yet, released to the general developer population. Some Q&A finishes out the session, as usual.

We've seen teasers about the plugin before on the GWT forums on Google Groups and some of the Eclipse project configuration files in SVN contain references to a "gwtNature" in a com.google package. This presentation went a tantalizing step further and included actual demonstration of the plugin's JSNI syntax highlighting and refactoring abilities, among other things. I wish we could get our hands on an early release version of the plugin, even an alpha or pre-alpha quality, just to try it out. So far, I haven't seen any firm commitment to a release date.

Several other informative presentations on GWT were given at Google I/O back in May. The GWT sessions are listed in the "APIs & Tools" track. The live sessions were a pleasure to attend and I highly recommend the videos, which I've referred back to on occasion since.

I make this look good

There is beauty, maybe even art, in code.

A well written piece of code flows smoothly. It is thoughtfully designed, observes proper encapsulation and variable scoping, and possesses a cohesive purpose. And it is well formatted. Lines are indented in the proper places so that the eye can easily follow the path down into nested constructs -- if statements, loops, anonymous inner classes, etc. -- emerging again into the main line of processing. Not only the structure, but the meaning and intent of the code are important in its visage. Variable names are concise without being cryptic, descriptive without being verbose. Capitalization and underscores are consistently applied where (and if) appropriate. Consistency in these elements makes code easier to read, just as proper spelling and grammar make the written word more comprehensible. Being able to tell what code means and what it's supposed to do at a glance is thrilling to a coder. If something thrills you when you look at it, isn't it beautiful? Good code looks good. Sometimes the beauty comes from the simplicity of the code and other times from taking a very complex algorithm and rendering it in a longer and more eloquent fashion which makes the complexity melt away as each line leads you by the hand through the intricate process.

The appearance of code is something that's been on my mind recently. In a discussion today on the GWT Contributor forum, I argued that certain proposed coding style guidelines disturbed the Feng Shui of code without adding value. It may not have been my most substantive argument of all time, but I think there's something of importance in that pithy reference to what many in the western world may think of as nothing more than an interior decorating fad. Each character in each line of code should blend harmoniously and purposefully with its surroundings. Even an extra space, or lack of one, could alter the careful balance of code without necessarily impacting its function. But that loss of balance harms the readability of code. Code that is less readable is less beautiful.

I always try to write my code in a way that appears neat and tidy. I've looked back on code I wrote years ago and thought to myself "damn, that's sexy." Sometimes (maybe more often, depending on how far back I wrote the code in question) instead I wonder what the Hell I was thinking when I wrote it. But most of those head-scratchers can be polished into more appealing code. It could simply be that my personal preference in matters of style and convention have changed, and applying those new preferences is all it takes.

Like fine art, music, and other things in which beauty can be found, beautiful code is often a matter of taste. But whether you prefer C++, Java, Lisp, .NET, Ruby, PHP, ColdFusion, Lua, or any of the myriad other varieties, take a moment sometime to appreciate the code. And please, for the sake of the code-lover who comes after you, strive to make beauty in your work -- appearances count.

Introducing GwtCompilerTask

One of the first things I did when I started working with GWT was to figure out how to compile an application. I don't mean running the myApp-compile.[cmd|sh] that the setup utilities create. I mean ripping apart those compile scripts and writing an Ant script to do it. Here's how in a nutshell:

<target name="gwtc">
  <java 
      classname="com.google.gwt.dev.GWTCompiler" 
      classpathref="classpath.gwt" 
      fork="true">
    <arg value="com.foo.gwt.myapp.MyApp" />
  </java>
</target>

But this past weekend, I found I needed something a little more powerful. I was writing a build target to compile all of the demos in the GWT Incubator. I wanted to avoid having to explicitly list all of the demo modules and use a <java> task call for each one. The best way to find demo modules seemed to be starting from the demo source root directory and scanning for .gwt.xml files. Ant has a <foreach> task that will do just that. The problem then was that <foreach> identifies the module files (com/foo/gwt/myapp/MyApp.gwt.xml), but GWTCompiler requires the logical module name (com.foo.gwt.myapp.MyApp). Ant is great for manipulating files, but it doesn't have much to offer in the way of runtime String manipulation. The solution? Leverage a more full-bodied programming environment. Like Java.

So I wrote a custom Ant task to handle invoking the GWTCompiler and to bridge the gap between the module's file name and it's logical name. Here's what the demo build looks like:

<target name="gwtc" depends="compile">
  <!-- define the GwtCompilerTask -->
  <taskdef 
      name="gwtc" 
      classname="com.google.ant.GwtCompilerTask">
    <classpath>
      <path path="${project.bin}" />
      <pathelement location="${gwt.dev.jar}" />
    </classpath>
  </taskdef>

  <property name="gwtc.vm.maxMemory" value="512m" />

  <!-- gwtc supports compiling with moduleName or moduleFile.
       Use of moduleFile requires setting src so that the 
       logical module name can be determined by comparing the
       moduleFile path to the source root path. vmMaxMemory
       sets the -Xmx VM argument. -->
  <gwtc src="${gwtc.src.dir}"
        out="${gwtc.out.dir}"
        moduleFile="${gwtc.module.file}"
        style="${gwtc.js.style}"
        vmMaxMemory="${gwtc.vm.maxMemory}">
    <!-- gwtc supports nested classpath -->
    <classpath>
      <path path="${gwtc.src.dir}" />
      <path path="${project.src}" />
      <path path="${project.bin}" />
      <pathelement location="${gwt.user.jar}" />
      <pathelement location="${gwt.dev.jar}" />
      <pathelement 
        location="${gwt.tools}/lib/w3c/sac/sac-1.3.jar" />
      <pathelement 
        location="${gwt.tools}/lib/w3c/flute/flute-1.3.jar" />
    </classpath>
  </gwtc>
</target>

<target name="build.demos">
  <property 
      name="demo.src.dir"
      value="${project.root}/src-demo" />
  <property name="demo.out.dir" value="${project.root}/demo" />
  <property name="demo.js.style" value="PRETTY" />

  <!-- Scan for any file under src-demo ending in .gwt.xml. For
       each file, invoke the gwtc target. The full path to the
       file is passed to gwtc target as gwtc.module.file -->
  <foreach target="gwtc" param="gwtc.module.file">
    <param name="gwtc.src.dir" value="${demo.src.dir}" />
    <param name="gwtc.out.dir" value="${demo.out.dir}" />
    <param name="gwtc.js.style" value="${demo.js.style}" />
    <path>
      <fileset dir="src-demo">
        <include name="**/*.gwt.xml" />
      </fileset>
    </path>
  </foreach>
</target>

GwtCompilerTask can also be used for building regular single-module GWT applications. GwtCompilerTask is available in the GWT Incubator. You will need to build gwt-incubator.jar from trunk at revision 1133 or later.

Taking Command of Your GWT Application

I love interfaces in Java. When used properly, they make for code that is highly cohesive and loosely coupled. Classes that implement interfaces or declare method parameters with interfaces are easier to reuse. One of my favorite interfaces in the Google Web Toolkit is Command. Command has one method:

public void execute();

That's it. A Command can be executed. Command, like the HasValue interface in the GWT Incubator is a very simple interface. So often in programming it is these simple building blocks that give us the greatest opportunities. Let's build on Command and another basic component, Button.

public class CommandButton extends Button {
  public CommandButton(final String html, final Command command) {
    super(html);
    addClickListener(new ClickListener() {
      public void onClick(Widget sender) {
        command.execute();
      }
    });
  }
}

There. Now we can create Buttons that execute a Command when clicked. If your application uses a lot of Buttons, that saves a lot of code for adding ClickListeners. And the code you might've put in those anonymous ClickListeners is now safely encapsulated inside a reusable Command class. I use some variation on CommandButton in almost every application I build. Here's another one I use:

public class CommandLabel extends Label {
  public CommandButton(final String text, final Command command) {
    super(html);
    addClickListener(new ClickListener() {
      public void onClick(Widget sender) {
        command.execute();
      }
    });
  }
}

Almost identical to CommandButton, isn't it? This one just uses a text Label instead of a Button. You can do the same with an Image if you want (and I have). I even wrote one with a ToggleButton.

At some point, I'm going to find the time to start a library project with all of these simple Widget extensions.

public int getValue()

This week I committed my first change as a member of the GWT Incubator project. That change was to introduce an interface, HasValue. This by itself is no monumental achievement. The interface is very simple:

public interface HasValue<T> {
 T getValue();
 void setValue(T value);
}

But this simple interface will facilitate the creation of more complex features such as libraries for data binding or validation.

HasValue provides a layer of abstraction around Widgets, allowing access to the underlying data model through a common API. HasValue works with simple Widgets and data types as well as complex data types and composite Widgets.

Consider two Widgets (both from the Incubator) for selecting a Date value:

DatePicker picker = new DatePicker();
DropDownListBox<Date> dropDown = new DropDownListBox<Date>();

Both implement HasValue<Date> so you could, for example, write a BirthdayValidator that ensures the selected Date is before today. This validator would have a HasValue<Date> parameter and could validate either of the two Widgets above, or any other Widget implementing the HasValue<Date> interface, without additional coding to support those Widgets.

Right now HasValue exists only in the Incubator, but there is support for including it in the next GWT release, version 1.6. Hopefully that release will see HasValue implemented by most of the standard GWT Widgets. In the mean time, I would love to see how GWT application developers use HasValue in their own projects. If you haven't worked with the Incubator before, you'll want to setup the project locally so you can build the latest features. Here's where to start:

Setting up the GWT Incubator project