Something Pithy (Steve Donie)

Why a culture of quality matters

2009-04-27T19:12:17Z

My company just recently started using ADP for processing our paychecks. So far, the core functions that ADP provides seem to work well, in that I do indeed get paid regularly. However, they have also provided us with access to their 'web portal' (beware of portals - they may lead to another world!) that allows me to do things like update my w-4, change my address, or download PDFs of paystubs. Recently we have also started using other ADP provided websites for other HR-related functions including recruiting and a very badly done company directory.

The developers on my team have had a great time pointing out just how bad the usability on these sites are, and mentioned it to our HR person. Our director of HR is a wonderful person, and is doing a fantastic job for us, by the way. She said that she could pass our concerns along to ADP, but that we should document some of the problems. That is where I got stuck. There were so many problems, I wasn't sure where to start. This morning, I started.

The task at hand was to acknowledge that I had read a certain company policy. I got email from HR:

In order to meet ERISA requirements, all employees eligible for one of our plans (regardless of participation) must acknowledge receipt and understanding of the document – Plan Documents, Certificates of Coverage, Summary Plan Descriptions.
I realize this may not be the highlight of our day, but it is an extremely important step in keeping our plans compliant.
Here are the easy steps which should only take a couple of moments.
Log on to our intranet site at www.portal.adp.com
Click on to the Resources tab – In the first column under Company Policy – follow instructions to acknowledge.
Feel free to contact me with any questions or concerns regarding this document or logging on to the site.

Sounded easy enough. But it didn't seem to be doing anything when I followed the instructions. Every time I clicked on the link, all I got was a PDF downloaded to my machine, and no indication that I had 'acknowledged' anything. So we called HR for help. Part of the problem is that the website is explicitly "Internet Explorer 6.0 or higher" only. I typically use Firefox. So I switched to IE. Still not figuring it out. After I finally got through the 4 step workflow, I decided it was time for some feedback.

Here is the email I sent back to HR.

As a general indication of the lack of quality in ADP websites, I found several bugs in the "acknowledge this policy document" workflow:
1. Step 1 is to find the correct link to click. You are presented with 2 options - a very large link to a PDF document, or a very small link 'click here to acknowledge'. Basic web and usability guidelines say that you should use larger targets for more important operations, and that using the word 'here' or 'click here' as a link are bad practice. (see item #2 at http://www.useit.com/alertbox/designmistakes.html Jakob Nielsen is a well-known author of several books on usability.)
2. When clicking on the correct link, a page is presented that only works in Internet Explorer. Several of my co-workers don't use Windows at all, so using Internet Explorer requires them to go to another machine.
3. When the page is presented, Internet Explorer warns that some items are the page are secure and some are not. I would call this a bug.
4. When clicking on the correct link in Internet Explorer, the document to be acknowledged is presented, but the "acknowledge" button is hidden at the bottom.
5. After clicking the acknowledge button, a dialog is shown that has wording including a font change tag (I don't have the exact text here, but it was something like "You have acknowledged that you have read the <font size="4">Policies of acknowledgment </font>...."
6. The dialog just mentioned ends with the phrase "Click OK to continue?" and the dialog has 2 buttons, OK and Cancel. It is unclear what the system would do if one pressed Cancel.
7. The system indicates success by removing the ability to read the document you just acknowledged, rather than by listing all the company policies and indicating those that had been acknowledged and those that had not.
This is sorta why I haven't provided any feedback to ADP yet. There are so many problems, and my gut tells me that the company does not have a culture of quality, and any feedback I provide will do little or nothing to affect the culture. It may help in a few small areas, but the real problem is the company culture that allows things like this to ship.

If anyone from ADP happens to read my blog, I would welcome comments on what the company culture is really like. I could be wrong.

Updated Release Burndown Spreadsheet Template

2009-03-26T22:20:00Z

I mentioned this release burndown template in one of my earlier posts, but I have updated it quite a bit since then, so I am re-posting the link, along with some instructions.

Download the spreadsheet

Here are the instructions - which are also in the spreadsheet itself.

This spreadsheet was evolved over several years, and incorporates work by Stefan Niejnhuis and Henrik Kniberg. It has been optimized for the process followed by a specific team, and while it can be used as a starting point for your own work, it should probably be adapted to fit oyur particular needs. I will try to describe some of the assumptions made so that customizing it can be done intelligently.

Basic usage is to start by editing the Work Remaining sheet. Set cell A2 to set your starting date. It will automatically fill out the remaining dates. Note that my team uses one week iterations that start and end on a Tuesday. Cell A2 is therefore assumed to be a Monday. Next, switch to the Stories sheet and start filling in one row for each story. You have to manually fill in the Code#, which is a unique ID. I just use incrementing numbers. The Buttons for Code# and Importance sort the stories. I typically keep the stories sorted by importance, but when adding a new story I sort by code # so I can figure out what the next story number is. For each story, fill in Story name, size (T, S, M, L) which automatically fills in the points column based on the lookup on the lookups sheet. Fill in importance - these should be unique numbers, with the most important things having the highest numbers. Typically I start by using 100, 110, 120, etc. and going up, leaving gaps to make it easier to squeeze new items in between existing items. It is very important that items have unique importance numbers. Fill in the created date and perhaps comments. Stories that have code numbers but no estimates get highlighted in yellow. Stories that are estimated but not prioritized also get highlighted in yellow.

For your day to day work, you will start by deciding on an arbitrary number of points you think you can do in one iteration. Sort the spreadsheet by importance, and start filling in the iteration column with '1' and watch the 'points this iteration' cell until you reach your limit. As the iteration progresses, mark stories complete by filling in the 'date completed' column and putting a 'y' in the 'Completed' column. If you decide to cut a story from this project, just put 'cut' in the completed column. If you want to postpone a story to some future release, you can put 'future' in the completed column. Each day after standup, update the work remaining page. Look at the numbers at the top right of the Stories page (Completed and remaining) and transcribe those to the correct row on the 'work remaining' sheet. Watch your graph grow automagically. One thing that I do is to use Excel's built-in 'publish automatically when saving' functionality to publish the sheet and the graph to some location on my intranet.

For following iterations, you will end the iteration by seeing how many points you completed over the past iteration (see the "Work Remaining" page, "5 day moving average velocity" and multiply that by 5. Assuming your team has the same capacity as the previous week, that is how many points you can commit to for the next iteration.

On the Stories page, there are three buttons at the top left. The first is "Generate Index Cards". Pressing this opens a dialog with a few options that should be self-explanatory. When you press the 'generate index cards' button in the dialog, a macro will start that deletes the old cards page, then copies relevant information from the stories page into a new cards page, using the CardTemplate page as a template. Printing these on 8 1/2 x 11 paper using the current template will give 2 cards per page. We just cut those in half and track the weekly iteration on a bulletin board. I'll blog about that sometime - keep your eye on http://stevedonie.lostechies.com/

The other buttons on the stories page are "compact view" and "expanded view". These are basically toggles between two different views. I use the compact view most often. When editing stories, I will switch to expanded view to make it a bit easier to type in the story notes and 'how to demo' sections.

Don't get too obsessive about it - the burndown is not the project, it is just a tool.

Note that this spreadsheet is a release-focused (rather than iteration-focused) burndown that has been created for the style of project that we work on, and may not be the best thing for you or your company. This was developed using Excel 2003, but is known to also work with Excel 2007. Also note that it does have macros, and that it is not signed. It comes with no warranty, expressed or implied. Use at your own risk. Your mileage may vary. Batteries not included. Do not mock happy fun ball. Enjoy!

Houston Alt.NET conference

2009-03-20T16:04:00Z

Just in case you haven't heard, the Houston Alt.NET conference is coming up in 2 weeks - there is still time to register.

http://houston.altnetconf.com/home

Now, I am primarily a Java developer, but I attended the first Alt.NET conference in Austin a couple of years ago, and what was 'sort of' the 2nd Alt.NET conference in Austin (aka KaizenConf). Both were just awesome events. The people that show up to these sorts of events are all people who are passionate about the craft of software and who take the time out of their lives to be continuous learners. There should always be a balance between work and life - I believe very strongly in spending lots of time with my family. But if you possibly can, I highly recommend coming. You won't be disappointed.

Looking for a new team member

2009-03-18T04:42:00Z

I have an immediate opening for a software developer. Level of experience is not my primary criteria. I am more interested in good object oriented design skills and a willingness and ability to learn and work in a team environment.

About the company: Drillinginfo is a smallish company (about 100 people). We gather data on the domestic oil and gas industry and put it all into a big database with a web front end with search, maps, etc. We sell subscriptions to companies ranging from 1-2 man shops all the way up to ExxonMobil. In the past few years, we have been growing at an annual rate of 30-40%. This year we expect that to be lower, but still growing. The company is about 9 years old, is privately held (no venture capitalists telling us what we should do this quarter), and has been profitable for several years. We offer a comprehensive benefits package including a 401K with match, company-assisted health insurance, vacation, etc. The company is headquartered in Austin, TX, on the south side of town near Loop 360 and MoPac. We have a great office space with views of downtown, direct access to the Barton Creek greenbelt, an exercise room, covered parking, and free lunch once a week.

About the team: I lead up the data integration team that writes the loaders bringing new data into the system. The team is 5 people, and we work in a large team room. We believe very strongly in the principles described in the agile manifesto, and use most of the XP technical practices along with Scrum project management practices. The software is 99% Java, with some groovy used in a few select places. We use a lot of open source tools and believe in contributing back to those projects where we can. The great thing about working here are the smart people you'll get to work with, the camaraderie of the team and the company, and the technical challenges we run into daily.

Refactoring project update #2

2009-02-18T15:35:20Z

So, work on the refactoring project continues. We had a meeting with some of the key stakeholders, and came to an agreement between everyone that we should strive for several small releases. Our first release goal is fairly modest, and actually removes some little-used functionality in the system. We're starting by setting up an end-to-end automated smoke test of the system. All of this is being automated using Hudson as our CI server. We approached the problem sort of like we would TDD, but in a larger context. Step one was to imagine what we wanted the end result to look like. Something like "Whenever a code change is checked in, we want to create a deployable packaged loader, save that artifact somewhere, then run a smoke test of that packaged artifact". That led to us looking at the packaging and artifact repository system we had been using. We had been using a system very similar to what I described in a post at my old blog in which built artifacts were checked into subversion. There were some issues with that system, and we've been wanting to move more towards a maven style of managing dependencies and build artifacts, so we spent a day or so getting that set up. On a side note, maven can be a pain in the butt.

So following up on my previous post regarding Hudson and CCTray, we now have all good builds!

Our next step is to get some FitNesse tests around some of the data validation aspects of this loader. The current loader is a bit rigid about how it does things - it uses exceptions for flow control (blech!) and just dies completely if there is any problem with the incoming data. The folks who run the loader have found this 'feature' a bit annoying. They run the loader and after running for an hour or so, it will die because of one problem. One of the new features we are adding is the auditing system we have developed for our previous loader projects, so problems get recorded in a standard format, but the loader can continue on with its work, loading the items that are valid, and skipping those that are not. The audit report is then sent back to our data entry group so that they either add to or correct the incoming data. We'll also set up some additional validations on the data entry side to catch the problems earlier.

CCTray with Hudson

2009-02-09T15:42:12Z

Hurrah! Eric Anderson did some research this weekend and we now have our CCTray lights and speech working again!

Back when we were using CruiseControl.java for our builds, I submitted some patches to the CC.net project to allow the CCTray application to properly control X10 build status lights, and also to support text-to-speech for announcing build status changes. We really liked having that. We had a couple of lava lamps set up in the center of the team room to show the status of our builds - that allows us to see the status of the build when we come into the room. The speech (It says things like "The Well Logs Refactor project has started building" and "The Well Logs refactor project reports Yet another successful build") helped us stay hyper-aware of the state of things. Having a periodic announcement that a build is starting reinforces the 'pace' of the team.

So, when we started using Hudson for our build server, we were disappointed that the rss format it shows didn't work with Hudson. I looked around a bit, and while Hudson has its own 'tray' application, it didn't do the stuff I wanted. I spent a little time looking at Hudson source code, thinking about writing a plugin to support the CC xml format. I looked at doing another patch for CCTray to support reading the Hudson RSS or XMl or Atom feeds. But I never got very far on those projects. Getting actual project work done was always a higher priority.

But now it is working again, and there was much rejoicing in the team room this morning. In order to spread the word that Hudson does output CCTray compatible xml, I am writing this post. The magic URL to enter into CCTray is http://hudsonserver:port/cc.xml and is described more fully on the Hudson Wiki.

Here is a picture of our newest build light setup:

Yes, the build is broken! We're creating a new smoke test at the moment, and fighting with Maven configuration. We're also going to neaten up the cords Real Soon Now.

Here are the three X10 modules on the wall. Pretty ugly. Might be a weekend project to mount the modules inside the light fixture, so we can just have a single cord coming out.

Finally, here is a pic of the team room - we have a long rectangular room, and mounted the lights on the short wall that is off the left edge of this photo.

Next thing for me will be a much simpler patch to CCTray to support three lights in X10. Currently it only supports 'green' and 'red'. I want to make it so that the yellow light also comes on while a build is in progress. So the color scheme would be:

Green only - all builds are good, nothing in progress
Green + Yellow - all builds are good, build in progress
Red only - at least one build is broken, nothing currently building
Red + Yellow - at least one build broken, build in progress.

Starting a new project - refactoring an existing system

2009-02-05T20:07:30Z

Well, the team is starting on a new project. Our company is in a situation that is probably familiar to many people. We have some existing code, we have data in the database, everything 'sorta' works. But now we want to add some new functionality, and there is fear. Fear that changes to the existing system will break things that currently work.

So far we have spent about a month trying to figure out the existing system. I spent some time meeting with developers who have worked on the existing code, spent lots of time meeting with the product owner, and lots of time looking at the code. It is definitely going to be a tricky problem. One of the most difficult things for me has been trying to work this project in an agile method. We have created stories and added them to a backlog, but a lot of them are very vague - "Understand and document Document table" is one example.

Here are some of the challenges we have already identified:

The whole system is basically a document management system that stores scanned images (Well Logs) and data about those well logs.
The system is comprised of loaders that read data coming from somewhere else that evaluate the data and then store the meta data in the database and the files on a fileserver.
Right now, there are at least 3 loaders in active use, each of which is similar to the others, but with a lot of copy-paste code reuse.
The database schema has a concept of a generic document, but has been 'polluted' over time with well log specific columns.
The is very little of the SOLID principles in use - we have jsp's that do presentation, database queries, and filesystem operations, for example.
There are very few tests of the existing system.
The document management system is fairly central to the company, and manages much more than just the well logs, so any changes made for well logs have to take other documents into account also.
There is a pricing system in place that charges customers for downloading the logs (naturally, with duplicated logic in various places).

One key thing we are trying to keep in mind as we work on this project is that we want to deliver business value as quickly as possible, but we also want to improve the system and reduce development effort, as we are fairly certain that there will be additional features and datasets to process in the future. So we're trying to strike a balance between a full rewrite of large parts of the system and shipping something that provides value quickly.

I'll keep writing as we progress on this project. It has definitely been one of the more difficult things I have ever taken on. I'd welcome anyone's feedback on how they have taken on similar projects.

Break out the popcorn, it's movie time!

2008-11-25T20:46:00Z

My team just finished up our most recent project - we spent about 4 months writing what we call a loader. It is basically an automated process that loads data from one database into another.

Over the past 2 years, as I have worked through several projects, I have created a spreadsheet that I use to keep track of our projects. One of the things it does is generate a release burndown chart. To help keep the rest of the company aware of our progress, I print that out every day or two and post it to our team room door. I also have Excel auto-publish the sheet, including the graph, to a webserver.

As we finished up, I decided it would be fun to see what the chart looked like in animated form. I was pretty pleased with the outcome of the project as we had been able to do several Good Things:

We were able to maintain a very consistent overall average velocity.
We were able to give the product owner information he was able to use to make decisions on what features to include or cut, based on a desired release date.
We were able to hit a release date without drama.
At the end of the project as we discovered problems with the data, our analysis, and the program itself, we were able to show how various decisions would affect the project.
It helped us to prevent feature creep - we had established credibility in the overall accuracy of our estimates and in our velocity.

So without further ado, here is a 6 minute analysis of the project using an animated burndown chart:

(Please visit the site to view this media)

(If you don't see the video embedded above, use the link to go over to YouTube)

One of the good things that has come from doing this is that when I first sent out a link to it inside the company, several people came up to me afterwards to say that they had learned a lot from it. One of the best comments was from our director of marketing. He said that he finally understood why I had been posting these on my door.

Educating everyone in the company about the benefits of agile software has been one of my most difficult challenges, and it feels like we are really making progress on that front.

For those interested, the tools used to create this are an Excel spreadsheet I use for burndowns, Windows Movie Maker for putting the movie together, Paint.net for doing modifications to various frames in the movie, Audacity for sound recording and editing, and Context programmers editor for editing the script. Basic process was to start at the end of the project and copy the graph from the spreadsheet into Paint.net and then save it as a .png file with a filename like yyyyMMdd.png. I then imported all the .png files into Windows Movie Maker and dropped them on the timeline after setting the default time for photos as short as possible. Watching the movie a couple times gave me some ideas on the script, so I started writing the script, deciding that I wanted to start at the end, rewind backwards to the beginning, and then move forward. As a wrote the script I saw that I was going to need some explanatory graphics, so I worked in paint.net to create the grayscale versions and the color-highlighted versions of some of the frames. After some final tweaking of the script, I used Audacity to record myself reading it. I did a little bit of editing to remove some pops and one or two places where I had to re-read something. After writing down timing down information on the script, I went back into movie maker to add frames and size everything to match up with the audio.

I've made the spreadsheet template available - this is the first time I have made it public, and I don't currently have any documentation for it other than what is in this post.

Basic usage is to start by editing the Work Remaining sheet. Set cell A2 to set your starting date. It will automatically fill out the remaining dates. It is up to you to factor in vacations, etc. Next, switch to the Stories sheet and start filling in one row for each story. You have to manually fill in the Code#, which is a unique ID. I just use incrementing numbers. The Buttons for Code# and Iteration sort the stories. For each story, fill in Story name, size (T, S, M, L) which automatically fills in the points column based on the lookup on the lookups sheet. Fill in importance - these should be unique numbers, with the most important things having the highest numbers. Fill in the created date and perhaps comments. Stories that have code numbers but no estimates get highlighted in yellow.

To start the project, just decide on an arbitrary number of points you think you can do in one iteration (we use 1 week iterations). You've sorted by importance, so start filling in the iteration column with '1' and watch the 'points this iteration' cell until you reach your limit. As the iteration progresses, mark stories complete by filling in the 'date completed' column and putting a 'y' in the 'Completed' column. If you decide to cut a story from this release, just put 'cut' in the completed column. Usually I will update the burndown right after standup. Look at the numbers at the top right (Completed and remaining) and transcribe those to the correct row on the 'work remaining' sheet. Watch your graph grow automagically. Use the built-in 'publish automatically when saving' functionality to publish the sheet and the graph to some location on your intrawebs. Bask in the adulation your peers and managers will heap upon you. Don't get too obsessive about it - the burndown is not the project, it is just a tool.

Note that this spreadsheet is a release-focused (rather than iteration-focused) burndown that has been created for the style of project that we work on, and may not be the best thing for you or your company. This was developed using Excel 2003, but is known to also work with Excel 2007. Also note that it does have macros, and that it is not signed. It comes with no warranty, expressed or implied. Use at your own risk. Your mileage may vary. Batteries not included. Do not mock happy fun ball. Enjoy!

newest Los Techie

2008-11-21T19:48:00Z

Thanks to all the Los Techies for inviting me to join the group. For those who haven't met me, I am software developer and team lead in Austin Texas. I currently work for Drillinginfo, a smallish company that gathers data about the domestic oil and gas industry and then sells it through a subscription-based website. I manage the Data Integration group, consisting of myself and 4 other developers. I've been here about 2 years, and have been working on converting the company from 'slash and burn' development to a more disciplined Scrum/XP process.

When I'm not writing software, I take photos, do woodworking, and play with my kids. I've recently started learning Tae-Kwon-Do with my youngest daughter, which has been really cool.

I have been blogging on and off since 2003 - first on blogger (http://stevedonie.blogspot.com/) and then on my own server (Something Pithy at http://donie.endofinternet.org/pebble/Steve/ ). At the Austin Tech book club the other day I mentioned that I was having some technical difficulties with my blog, and John invited me to start posting here. I'm really excited to be a part of this great group of developers.