PhatBoyG

Managing Long-Lived Transactions with MassTransit.Saga

2008-08-29T04:48:15Z

One of the first applications we built with MassTransit provides messaging for a long-running transaction started by an application submitting a request. The request is formatted into a X12 envelope and sent to a web service. An intermediate response is returned (a X12 997) with a correlation identifier for the request. Another web service is polled for the response, which can be the result or an indication that the request is still pending. When the response is received, the X12 document is translated and stored in the database. Finally, the user is notified that the transaction is complete and the result is displayed.

As work proceeded on this application, I started to recognize the need for something to coordinate the different steps in a transaction involving multiple loosely-coupled services. Due to the duration of these transactions (the example above can take anywhere from three to sixty seconds), it is unreasonable to keep a single System.Transactions style transaction open the entire time. I started researching how others approached the problem and found a couple of articles that helped. After reading Sagas by Hector Garcaa-Molrna and Kenneth Salem (© 1987 ACM, PDF) and the chapter on sagas in the upcoming book Practical SOA by Arnon Rotem-Gal-Oz (PDF), I started to think about how this could be implemented within MassTransit.

I should note that NServiceBus (another open source service bus) also supports sagas, but I purposely avoiding taking a look at how Udi Dahan implemented them. Once saga support in MassTransit is complete I plan to review the source for NServiceBus to see how the implementations differ. I spoke with Udi at ALT.NET Seattle and his writing has been both educational and inspirational. A lot of great discussions in the NServiceBus mailing list have been an excellent resource as well.

So after a few weeks of trying to flesh out the structure (using TDD, of course), I finally arrived at what I think will be a highly usable infrastructure for handling sagas. In the project MassTransit.Saga.Tests, I've created a test that simulates a user registering for a web site. The class for the registration is shown below.


public class RegisterUserSaga :

	InitiatedBy< RegisterUser >,

	Orchestrates< UserVerificationEmailSent >,

	Orchestrates< UserValidated >,

	ISaga< RegisterUserSaga >
{
	private string _displayName;

	private string _email;

	private string _password;

	private string _username;



	public RegisterUserSaga(Guid correlationId)

	{
		CorrelationId = correlationId;
	}



	public Guid CorrelationId { get; private set; }

	public IObjectBuilder Builder { get; set; }

	public IServiceBus Bus { get; set; }

	public Action< RegisterUserSaga > Save { get; set; }



	public void Consume(RegisterUser message)

	{
		_displayName = message.DisplayName;

		_username = message.Username;

		_password = message.Password;

		_email = message.Email;



		Save(this);

		Bus.Publish(new SendUserVerificationEmail(CorrelationId, _email));
	}



	public void Consume(UserVerificationEmailSent message)

	{
		// once the verification e-mail has been sent, we allow 24 hours to pass before we

		// remove this transaction from the registration queue

		Bus.Publish(new UserRegistrationPending(CorrelationId));

		Bus.Publish(new UpdateSagaTimeout(CorrelationId, TimeSpan.FromHours(24)));
	}



	public void Consume(UserValidated message)

	{
		// at this point, the user has clicked the link in the validation e-mail

		Bus.Publish(new UserRegistrationComplete(CorrelationId));

		Bus.Publish(new CompleteSaga(CorrelationId));
	}
}

At the top of the class, the messages consumed by the saga are specified. InitiatedBy indicates the message initiates a new instance of the saga. Orchestrates is for messages that are part of the saga once it has been initiated. All saga instances are identified by a Guid and all messages consumed by the saga should have the CorrelatedBy< Guid > interface. A saga must also implement the ISaga generic interface to allow certain properties to be set giving the saga instance access to the bus and the object builder.

Once the saga class is added to the bus (via the AddComponent method), any messages consumed by the saga will be dispatched to the saga instance. A generic ISagaRepository must also be registered in the container so that sagas can be persisted between messages. The saga dispatcher uses the repository to either load or create the instance of the saga. Since instances of the saga class are saved, the class can expect the members to also be persisted between messages allowing state to be retained.

There is still some work to be done, including a service to handle timeouts and retries. It will be up to the developer to handle any compensating actions that need to be taken in the case of a failure. Therefore, it is highly suggested that the saga also consume any Fault< T > messages that are published when a message consumer throws an exception -- particularly if the consumer is not part of the saga (such as an application or domain service).

The code is currently in the trunk and slated to be part of the 0.3 release.

Distributed Processing with MassTransit.Grid

2008-08-23T14:18:04Z

I was reading through the Xgrid documentation for OS X yesterday after reading an article on Integrating Xgrid Into Cocoa Applications. The article gave me some ideas and I decided to see what it would take to build a distributed processing system on top of MassTransit. The result is a new MassTransit.Grid namespace that includes support for building distributed task processing into an application. The following sections define the language used in the distributed task classes.

Distributed Tasks

A distributed task contains one or more subtasks that need to be processed concurrently across multiple systems. To create a distributed task, create a class that implements IDistributedTask. The input and output types for the subtasks must also be defined by the distributed task class.

public interface IDistributedTask< TTask , TInput, TOutput >
{

int SubTaskCount { get; }
TInput GetSubTaskInput(int subTaskId);
void DeliverSubTaskOutput(int subTaskId, TOutput output);
void NotifySubTaskException(int subTaskId, Exception ex);
void WhenCompleted(Action< TTask > action);

}

Subtasks

A subtask is an individual unit of work within a distributed task. Each subtask should be completely standalone and not depend upon the completion of any other subtask within the distributedtask. There is no attempt to execute the subtasks within a distributed task in order. A subtask has specific input and output types, each of which are defined by a class (POCO style). These input types are used to determine which workers are used to process the subtasks.

DistributedTaskController

To insulate the application from the details of coordinating the subtasks, a generic DistributedTaskController is used. This class is built from the class that implements IDistributedTask, along with the input and output types. Once created, the application can call .Start() to being processing the distributed task. The controller performs any initial identification of workers that are available to process the subtasks, along with the coordination to ensure that workers are not overloaded.

public class DistributedTaskController< TTask , TInput, TOutput >

TTask is the class that implements IDistributedTask, TInput is the subtask input type, and TOutput is the subtask output type.

Workers

To make it easy to create workers to handle subtasks, a default worker implementation is available. This worker handles the coordination with the DistributedTaskController, along with the delegation of the messages to the actual subtask worker. For example, a worker that accepts a GenerateFileHash object and outputs a FileHashGenerated object would be setup as shown:

public class FileHashGenerator :
ISubTaskWorker< GenerateFileHash , FileHashGenerated >
{

public void ExecuteTask(GenerateFileHash input, Action< FileHashGenerated > output)
{
string path = input.Path;

// do work here

output(new FileHashGenerated());
}

}

The worker can then be added to the container for servers that will be processing the subtasks using:

_container.AddComponent< FileHashGenerator >();
_bus.AddComponent< SubTaskWorker < FileHashGenerator, GenerateFileHash, FileHashGenerated > >();

This will register the SubTaskWorker for the worker as a message handler for the messages that are used on the transport to transfer the input and output data between the controller and the subtask workers.

Exception Handling

If an exception occurs in a subtask, the worker and controller leverage the built-in fault handling support of MassTransit to notify the distributed task that an exception has occurred. The controller will call the NotifySubTaskException method with the subTaskId and the exception that was thrown by the worker allowing the distributed task to determine the next course of action based on that failure. Options would include simply aborting the distributed task, fixing the input data and adding it to the end of the subtask list, or some other application-defined behavior.

Dynamically Adding Subtasks

To reduce the impact of setup time on the overall duration of a distributed task, it is not necessary to have all of the subtasks loaded before starting the distributed task. This also allows additional subtasks to be added based on the output from other subtasks. For example, a task to parse a remote file system may identify additional folders that need scanned for content. The distributed task could just add those folders to the end of the subtask list and they would be picked up by the controller. By allowing this, the distributed task is responsible for calling the delegate set by the controller to indicate that all of the subtasks have completed. The DistributedTaskController will then release any resources that were in use.

Sample in Unit Tests

A quick sample was built in the unit tests (MassTransit.Grid.Test) that shows an integer factoring service. The distributed task creates a bunch of very large integers and processes them as a distributed task between the workers that are available. Hopefully this demonstrates how the classes are hooked together since this was used to drive out the feature set.

Wrapping Up

This is just a brief introduction to the distributed processing capabilities that were added to MassTransit. There are likely some additional features to add that will hopefully be identified as the feature it put to use. Therefore, it is important to note that this feature is still in development and should go through some considerable testing before putting it into use in a production application. Any feedback is always welcome (including patches) so try it out!

MassTransit 0.2 Now Available

2008-06-14T04:20:45Z

We’ve dropped a new release of MassTransit today, version 0.2 is now available on the main page. There are several new features included in this release. It was great to get some feedback from people who have tried MassTransit, along with the evolution from discussions with Greg Young, Udi Dahan, Ayende, and others at ALT.NET Seattle.

A quick summary of the changes in this release:

Extensible Message Dispatcher

To make it easy to add new messaging patterns to the service bus, MassTransit has an entirely rewritten message dispatcher. The new code is structured in a producer/consumer style that lends itself to easy extensibility, allowing new features such as batch messaging and correlated messages without heavy lifting.

Component-Based Message Handling

In the previous version, handling messages required an object to subscribe to messages types passing methods to handle the messages. Now, a class can implement interfaces to support message consumption and the class itself can be added to the service bus. The service bus will then create objects to handle each message, removing the need to use the same instance for each message received. A class can handle multiple messages types, and can also indicate whether to receive all, selected or correlated messages by implementing the Consumes.All, Consumes.Selected, or Consumes.For interface.

Batch Messaging

The new message dispatcher includes support for message batches. Instead of having to correlate a batch of messages at the application layer, an object or component can consume a batch instead of each individual message. Details of how to use batch messaging are on the wiki.

Container Integration

We finally bit the bullet and started using a container. A new assembly called MassTransit.WindsorIntegration adds a default container derived from WindsorContainer that adds facilities to create ServiceBus instances. A new custom syntax was created to make it easy to configure multiple ServiceBus instances. All of the samples have been updated to use the new container to help understand the integration points.

Plain Old C# Object Message Objects (POCOMOs Anyone?)

The need to have all messages implement IMessage is gone, reducing the footprint of MassTransit in your application code. There are some new interfaces to handle things like correlated messages and batch messages, but those are only needed to use the built-in support for those message patterns.

Better Thread Management

To allow for more control over resources, a new thread manager has been added. While we haven’t exposed the thread configuration yet, a dedicated thread pool for asynchronous message dispatching should allow for more efficient message handling. We also took all the threading code from the endpoint and put it in the service bus, reducing the complexity required for new endpoints. In fact, the endpoint structure has also been redesigned to be more send/receive focused.

Publish/Subscribe Focus

The service bus now has a pure publish/subscribe architecture compared to the previous additional methods, such as Send() and Request(). For applications that need to send messages directly to endpoints, the endpoint now has a Send() method. The publishing of messages takes advantage of the same new subscription code, allowing all messages type information to be cached for better performance (avoiding the reflection penalty on each call to Publish). With the extensibility of the message dispatcher, it’s likely that remote endpoints may some day find there way into the dispatcher, resulting in a single dispatch engine for asynchronous publishing of messages as well.

Request/Reply

Requests are handled in an entirely new way, using a new fluent builder. The fluent builder allows the calling code to subscribe to any responses (directed via the Consumes.* interfaces, indicate whether an asynchronous callback should be allowed (for [WebMethod] style Begin/End usage, PageAsyncTask usage, or MonoRail asynchronous actions), and get a future object that can be used to complete the request upon receipt of a response. The responses are handled by the calling class itself, the future object is used to signal the operation complete which will release any waits or callbacks for the action.

Distributed Subscription Cache

A new distributed subscription cache (backed by memcached) is now available. This is mostly designed for high volume request/reply applications that need to add and remove a lot of correlated subscriptions and maintain a high level of performance. A load test will be added to the HeavyLoad sample soon, but it seems to hold up pretty well so far under regular testing.

Dashboard

Dru has been hard at working making an operations dashboard part of the core product. One of the big things about messaging systems is being able to assess the health of the endpoints at any time. The goal of the dashboard is to provide that single pane of glass to find out which endpoints are alive, what messages they are handling, and indicate any problems to the viewer. There are many plans for this, including the ability to remotely control the endpoints for dynamic adjustments to load handling and perhaps even remote service restarts.

Deployment

Dru has also been working on the deployment story, making it easier to deploy MassTransit into a production system. There is much love needed there, but I’m hoping to dig into it soon to see how things have gotten easier to manage.

I'm sure there are many more features that have been added under the covers. The main thing is that with this release (0.2) we're pretty happy with the API experience. The consumer code is easy to understand and implement, particularly compared to the earlier version. It is unlikely that we'll do another major overhaul to the interface like we did with this version.

So give it a shot, and blog about your experiences!

Enhancements to MassTransit (or Weekend of Coding)

2008-06-02T13:16:21Z

This past couple of weeks I've been putting some serious time into MassTransit. My primary goal is to improve the internal architecture and remove some of the MSMQ leaks into the infrastructure. Our original goal was to stick purely to MSMQ, however, as we got more into messaging systems we found that there are a lot of other transports with different advantages. For example, using ActiveMQ would make it easy to add integration with Java applications down the line. The problem at this point was all the code designed around MSMQ was making it difficult to support other transport types.

About two weeks ago, I started working on a completely new method of dispatching messages within the service bus. My goal was to support a pure producer/consumer model using a publish/subscribe pattern. With this in mind, I built a new message dispatcher that allowed for a new way of specifying message subscriptions. Previously to this change, a service interested in messages would do the following:

_bus.Subscribe<MyMessage>(MyHandlerMethod);
public void MyHandlerMethod(IMessageContext<MyMessage> context) {}

Because of this structure, there had to be an instance of the class in memory and it somehow needed to be started so that subscriptions could be added. The class also needed to be stopped so that it could remove any subscriptions from the service bus. Another goal was to be able to use an object builder to create objects as needed to handle messages. For example, we wanted to use Castle Windsor to dynamically build objects to handle messages and get all the injection benefits of the container.

To support this new style, I added some new interfaces and made it possible to register either an object or a class with the service bus. As an actual example, compare the original subscription client code to the new version:

Before the changes:

public class SubscriptionClient : IHostedService
{
	public void Start()
	{
		_serviceBus.Subscribe<AddSubscription>(HandleAddSubscription);
		_serviceBus.Subscribe<RemoveSubscription>(HandleRemoveSubscription);
	}
	public void Stop()
	{
		_serviceBus.Unsubscribe<AddSubscription>(HandleAddSubscription);
		_serviceBus.Unsubscribe<RemoveSubscription>(HandleRemoveSubscription);
	}
	public void HandleAddSubscription(IMessageContext<AddSubscription> ctx)
	{
		_cache.Add(ctx.Message.Subscription);
	}
	public void HandleRemoveSubscription(IMessageContext<RemoveSubscription> ctx)
	{
		_cache.Add(ctx.Message.Subscription);
	}
}

And now after the changes:

public class SubscriptionClient : IHostedService, 
	Consumes<AddSubscription>.All, 
	Consumes<RemoveSubscription>.All
{
	public void Consume(AddSubscription message)
	{
		_cache.Add(message.Subscription);
	}
	public void Consume(RemoveSubscription message)
	{
		_cache.Remove(message.Subscription);
	}
	public void Start()
	{
		_serviceBus.Subscribe(this);
	}
	public void Stop()
	{
		_serviceBus.Unsubscribe(this);
	}
}

The code just makes more sense and is easier to understand after the changes. In addition, you can also just call _bus.AddComponent<T>(); to register a type with the service bus and it will use the object builder to create an instance of the class to handle the message. If you're using a container like Windsor, you can specify the lifestyle of the object(s) there, either singleton, transient, or pooled -- depending upon your application requirements.

Also notice that there are various types of consumers supported, indicated by the interface used in the consuming class. Consumes<T>.All will deliver any message of type T to the consumer. Consumes<T>.Selected adds an Accept(T message) method to the class to screen any messages before removing them from the endpoint (at least with MSMQ, likely not the case with ActiveMQ).

The third option presently available is Consumes<T>.For<V> and adds support for a correlated consumer. In previous versions of MassTransit, Request/Reply was handled by using the transport message identifiers and setting a correlation identifier on the transport message. This leaked a lot of details into the service bus layer that were not pretty. Instead of using the transport correlation identifier for messages, we decided to add a new interface that messages can implement called CorrelatedBy<V>. This interface has a single method that returns the correlation identifier for the message -- and it is expected that the message body itself will contain the correlation identifier.

So now a request/reply pattern would look something like this:

class Controller : Consumes<Response>.For<Guid>
{
	public void Consume(Response message)
	{}
	public void Action()
	{
		_actionId = Guid.NewGuid();
		_bus.Subscribe(this);
		_bus.Publish(new Request(_actionId, someValue, someValue2);
	}
}

When the object subscribes to the bus, the correlation identifier is used to filter incoming messages so that only correlated messages are delivered to the object. This is cleaner from a interface contract perspective since you can look at a service and see what messages are produced and ensure that your controller implements all of the expected responses.

While working on these API changes, I also made a number of other changes including:

Messages no longer need to implement IMessage, plain old objects can be used
Removed all threading from the endpoint (asynchronous message dispatching is now handled by a thread manager in the service bus
Added a DispatchMode so that messages could be dispatched synchronously for unit tests

I also wrote a new sample called HeavyLoad to benchmark the performance of the bus when using various transports. A variety of message per second tests are performed to see how well the system can be expected to perform based on the type of messaging being done. Early tests on my system (Windows 2003 server in VMware Fusion) show MSMQ performance to be between 950-1500 messages per second (for a 300 byte message, persistent) and around 500 messages per second doing a correlated request/response with a single thread (but using the asynchronous dispatcher). If I were to rewrite the test to use multiple message send threads I would expect performance to increase somewhat since my load test is a bit naive at the present time.

At the same time, I managed to extend the subscription support to include correlated subscriptions. The only subscription cache that currently supports the extensions is the DistributedSubscriptionCache (which uses memcached to share subscription information across a distributed group of systems). The key goal here was to enable MassTransit to support a distributed request/reply architecture using publish/subscribe with correlated subscriptions to specifically route messages to their intended consumers. I plan to make heavy use of this in an upcoming project so I wanted to see it work.

In addition to all the changes, I also updated a few of the samples and made various tweaks to the infrastructure to make it cleaner. There are several more tweaks on the whiteboard that I'm hoping to investigate in the next week. Once those are done, full ActiveMQ support is up next including running the tests under Mono on OSX.

So a lot of changes since the 0.1 tag was put down a couple of weeks ago. I expect there will be some continued testing and tweaking this week as Dru seeks to understand all the changes that were made. While I've been doing this stuff, Dru has gotten a kick ass start on a new dashboard to monitor an application built using MassTransit. The goal there is to provide a single pane of glass view into the health of a system, including subscriptions, endpoint throughput, message counts, etc. We've got some cool ideas how to make the information available and hope to make this alone one of the cool features to help support distributed messaging applications.

If you haven't checked out MassTransit, you can get the latest source from the GoogleCode repository. There is a message board for questions, or feel free to contact Dru or myself with any questions.

VMware Fusion 2.0 Beta 1

2008-05-06T13:59:10Z

It looks like VMware has dropped a public beta of VMware Fusion 2.0 to the public. There are too many new features in this release to mention, so I will just share the link:

http://communities.vmware.com/community/beta/fusion

Be sure to check the release notes to see if they impact your work before upgrading to this beta release.

Items of interest I took away from the release notes:

Use Fn+M to simulate the non-existent insert key in Windows (hello R#)
Multiple display support for virtual machines (hello second monitor)
Improved compatibility with Visual Studio (not sure what this is yet)

Be sure to check it out!

Open Spaces In Practice

2008-04-30T04:04:59Z

After a great weekend in Seattle for the ALT.NET Open Spaces event (aside from a slight error in judgement on Friday night as depicted in this blog post), the two coworkers and I discussed how we could bring the experience of Open Spaces back to the team in Tulsa. We decided that instead of just giving a few talks about some of the things we took away from Seattle, we would bring the experience itself to the team.

At the end of our team meeting on Monday, we laid out some paper and pens and asked members of the team to write up topics that they wanted to discuss. It started a bit slow, but within minutes we had eighteen topics on the wall. The variety of topics was excellent, most of which targeted a different subset of the team. It was great to see the team come up with such a nice list of things for the team to discuss.

We then drew a grid of time slots on the whiteboard and asked the team to put their initials on the topics they wanted to attend. As the papers filled up, we started aligning them into the grid based on popularity and expected duration. We tried to keep all of them to an hour but some of the topics were given ninety minutes based on the content.

We had to negotiate on some of the topics to fit our time slots, and combined a couple of similar discussions into the same time period so that we could maximize our coverage over the next few days. Some topics were brought up that are timely to an upcoming release of our software, so those were given a slight priority in the schedule.

Overall, I was very impressed with the contributions by the team. Topics included things like "maximizing the use of Resharper", "What are unit tests", and "How can we test better?" With the large number of topics on the schedule, we're hoping that we can set aside a couple of days a week (maybe Tuesday and Thursday over lunch) to continue with more discussions. I'll be sure to post a follow up as things unfold over the next week.

ALT.NET Seattle Starts Tonight

2008-04-18T22:05:17Z

While I'm trying to keep my posts at pace with the event here in Seattle, it gets harder as more attendees arrive. Yesterday, Evan Hoff showed up and we took another trip down to Pike Place Market for some food and festivities. After an afternoon of good conversation, we went to Kells Irish Pub to meet up with Ayende and Udi (along with some other folks) to talk about software architecture.

The variance in perspective between different architects is always interesting, and this meeting did not disappoint. Recognizing the strong points in each design style was helpful by understanding the background of each approach. I'm looking forward to more of that this weekend.

The weather turned completely crappy today (technical term, sorry), as it is cold and wet. We still managed to run about and snag some chow before things get fired up tonight. A pretty solid crowd is overflowing the lounge in the hotel and you can tell there is some energy building.

We're about to head that way to get the final things wrapped up and organized. Look for more later tonight (or tomorrow if the talks get deep)!

ALT.NET Seattle Pre-Conference Warm Up

2008-04-17T06:21:20Z

I arrived in Seattle today to get warmed up for the ALT.NET Seattle conference this weekend. I came up a couple of days early to meet with a few people before the event started on Friday.

Once we landed, our first trip was to the Pike Place Market, which is right on the shore of the Peugeot Sound. We had lunch at a great chowder house, followed by a coffee at the original Starbucks coffee shop. After the food and drink stop, we walked around the market and stopped at an REI outfitters (like a Bass Pro without all the hunting gear) since Dru forgot his jacket in Kansas. As we left the store, the sun came out and it was really too warm for a jacket -- such is luck. So we got back in the car and drove to a mini-mart to get some Monster to fuel the next few days, after which we parked downtown and walked to the Westin to meet up with some guys.

When we got to the Westin, we met up with Nick Parker to talk about various open source .NET projects that are active. After a while, Greg Young joined us and shared his understanding of how messaging applications should work and gave us some tips from his experiences using various forms of message transportation. The discussions started off general and got more specific as we talked about particular scenarios within our applications.

Since Dru Sellers and I both work on MassTransit and both use it in our commercial applications, it was very helpful to get feedback on this key infrastructure component. What we found is that while we have a pretty solid start, there are a few clarifying points that will help elevate the platform to be even less complex while adding new functionality. We came away with a lot of good ideas (and some diagrams, which David Laribee never actually witnessed being drawn) that we will probably explore over the next day or two.

After a few beers and some good discussion, the MVPs kicked over to the evening party that MS had arranged, so Dru and I went to dinner and drove out to the hotel.

Tomorrow should be an interesting day as we try to lay down some of the things we talked about tonight into code. We also have a very exciting dinner planned for tomorrow night, but I'll save that for another post later in the week.

Preparing for ALT.NET Seattle

2008-04-13T19:36:45Z

While the MVPs descend upon Seattle for the 2008 MVP Summit, I'm getting ready to head to Seattle for the ALT.NET Open Spaces Conference this coming weekend. Many of the people attending the ALT.NET conference are also MVPs, so there are likely to be some tired folks by the end of this week(end).

Based on the experience I had in Austin at the first ALT.NET conference, my company has agreed to send me to Seattle along with a couple of coworkers. A fellow developer and our agile project manager are going along to share in the experience and join the conversation. As an organization, we've seen many improvements since converting to agile development (XP-style) and I'm hoping the conference will help identify some new ways to help us grow and educate the rest of the team.

From a project perspective, Dru Sellers and I are trying to wrap up a 0.1 release of MassTransit in time for the event. We're hoping to be able to share some of what we've done with others who are approaching a distributed application architecture, and even more importantly learn from others with similar experience so that we can improve MassTransit and add some new routing and management features. We're getting there a couple of days early to have some time to pair up and knock out a few last minute tweaks before calling it good.

I'm looking forward to seeing a lot of the friends I met at the first event, as well as making some new friends. I'm excited to have "Doc" facilitating the event, his coordination of the first event was an inspiration at the start of each gathering (regardless of how many beers were consumed the night before).

As the event unfolds, I'll be adding pictures to a web gallery for the event. Be sure to check it out (or add a subscription to the feed) if you weren't able to attend in person. I'm sure there will be plenty of tweets as well, along with an RSS feed overload after every gathering.

Tulsa School of Dev May 10, 2008

2008-04-04T17:09:40Z

It looks like the local Tulsa .NET Users Groups (TulsaDevelopers.NET) is planning to have a spring code camp on May 10th, 2008. It's going to be at the Tulsa Community College North Campus. I'm hoping that I'll be able to do a presentation on message-driven architecture at the event along with fellow MassTransit author Dru Sellers. We plan on refining some of the MassTransit architecture after talking with Udi Dahan and Ayende Rahien at the upcoming Seattle ALT.NET conference.

You can get more information at the Tulsa School of Dev 2008 web site.

Coding Tunes

2008-02-29T15:53:00Z

I got tagged by Max Pool of {codesqueeze} regarding my choice of tunes while coding.

For me, it's all about down tempo beats and no lyrics. And it started on DI.fm in the Chillout Channel.

I was listening to a mix that just really got the juices flowing called Ott Times. It was mixed by Morlack. The song lead me on a quest to find the artists in the mix, which included:

Shpongle
Ott
Younger Brother
Entheogenic
Shulman
Hallucinogen

Since then, I've managed to acquire almost every physical CD of each of these artists. I've put together a few of my favorite playlists that I enjoy while coding, chilling, or doing whatever. So that's my flavor, one that's worked for a while. I've since added some new artists to the roster, including Phutureprimitive and a few others. It's amazing what you can find just lying in bed searching through the iTunes music store on your iPhone.

Developing for Windows on Mac OS X

2008-02-17T01:56:03Z

This past week I setup a new development environment on my MacBook Pro. As I got into it, I realized that it might be a good idea to share the setup process with others who might be switching to a Mac. Unlike a lot of people that use Windows XP for development, I do all my .NET development on Windows Server 2003. I have always had a firm belief in developing on your target environment.

For this discussion, I will be using VMware Fusion to run Windows. I've always been a fan of VMware, and their Mac OS X version continues to impress. There are many slick features in VMware Fusion, including Unity mode and dual-processor support. I'm going to be installing on my MacBook Pro, which is a 2.4 GHz Core 2 Duo with 4 GB of RAM (using the Santa Rosa platform, for full 64-bit memory support). I'm going to install the 32-bit version of Windows Server 2003 since the tools are all 32-bit applications.

I highly recommend getting your installation disks as ISO files (from MSDN, of course). VMware can mount the disk images directly, and they load a lot faster than using the physical media. For the purpose of this installation, you'll need the following disk images:

Windows 2003 Server R2 with Service Pack 2 (disk 1, 2 is not needed)
Visual Studio 2005 Professional
Microsoft MSDN Library for Visual Studio 2005 (optional)
Microsoft SQL Server 2005 Developer Edition

You'll also need the following downloads:

Creating the Virtual Machine

To create your virtual machine, start up VMware. On the Virtual Machine Library window, select New. On the New Virtual Machine Assistant, click Continue. For Operating System, choose Microsoft Windows, for version, choose Windows Server 2003 Standard Edition. When choosing a name for your VM, I find it helps to pick something that reflects what development will be done. I have two, one called W2K3x2K5 and one called W2K3x2K8. As you might guess, the first has Visual Studio 2005 and the latter has Visual Studio 2008.

When choosing a disk size, be sure to give yourself enough room for your tools and applications, but don't go overboard. Before I rebuilt my VMs, none of them were over 14GB in size so I chose a size that would give a little room to grow. Now, you could pick something like 40GB and configure the VM to grow the disk file as space is consumed but I prefer to allocate the entire disk in 2GB chunks at creation to get the best performance. I don't usually use a lot of local space since all of my source code is kept in a remote Subversion repository.

The next screen is the Windows Easy Install. This is an easy way to get the operating system setup without any user intervention. Just make sure your install disk ISO is ready, and enter your name, password, and Windows key. Check the "Make your home folder accessible to the virtual machine" so that you can easily transfer data from OS X to Windows. I make the folder read only, since I don't trust Windows to write to my home folder. Once the Finish screen displays, uncheck the Start and Install now option since we want to make some tweaks before the OS install begins. You can go ahead and pick your ISO for the installation.

Once the VM has been created, go back to the Virtual Machine Library, pick your virtual machine, and click Settings. We need to tweak a few things before we let Windows install.

Battery - I let the guest see the battery status
Display - Uncheck since we are not using XP
Memory - I give the VM 1492 MB, but you can adjust to taste.
Processors - Two (2) virtual processors should be set for installation.
Network - NAT or Bridged are the only real options here. NAT is good when you don't need other machines to have access to the VM, but use Bridged if you want a direct link to the network. If you use NAT, you can get to your Mac from the VM using the .host name.
Sound - I like sound, I enable it.
Shared Folders - This should be fine based on how the VM was created.
USB - I like to be able to use my flash drive in the VM, so I enable this option.

Once the settings are tweaked, go ahead and fire up the VM. Windows should install without any user intervention and after a short time (the install runs super fast from the ISO) you should have a working Windows Server 2003 installation. Once you have it up and you are logged in, it's time to run Windows Update to get all the latest patches (there will be a ton of them). Go ahead and let them install, and be sure to get the optional .NET 2.0 and .NET 3.0 packages as well.

Now that we have Windows setup, it's time to take away all the things we don't need.

Disable automatic updates -- we can do that when we need to do it.
Disable remote access into the machine (no Remote Desktop or Remote Assistance).
My Computer, Properties, Advanced, select Adjust for Best Performance for Memory and CPU.
Set virtual memory to a fixed size of 1GB.
Advanced Startup and Recovery - Uncheck all reporting and set Debugging Information to none.
Disable error reporting and turn off critical error notification

In the control panel, we need to tweak a few things as well:

Set your time zone and turn off automatic time synchronization.
Turn off the screen saver, set the background to none, and make sure Windows Classic is selected for the theme.

In the registry, this tweak really decreases disk access in your VM:

HKLM\System\CurrentControlSet\Control\FileSystem
Add a new DWORD named NtfsDisableLastAccessUpdate and set it to 1.

I also disable the following services:

Automatic Updates
Error Reporting Service
Help and Support
Windows Time
Wireless Configuration

Remove the following files:

All the $whatever$ directories in C:\Windows except for the $hf_mig$ directory.

Add Operating System Components

Use Add/Remove Programs, and select the Add/Remove Windows Components option.
Add IIS (including ASP.NET support)
Add Message Queueing (if you plan to use it)
Remove Enhanced Internet Explorer Security
Install any extra fonts you use for development, such a Consolas.

It's probably a good idea to run Windows Update again after adding these features since they've likely been patched.

Back Up Your New VM

Now that we have a nice clean system, I recommend you shut down the VM and make a backup copy. Simple copy the Documents/Virtual Machines/YourVMName to an external drive so you can easily create new VMs without doing the Windows install again. If you do so, I recommend using SysPrep to reseal the operating system so all new SIDs are created. Do not make a backup for the purpose of closing after installing SQL Server 2005 since it goes crazy if you change the machine name after it is installed.

Software Installation

I recommend installing Visual Studio 2005, followed by SQL Server 2005. For Visual Studio, if you aren't going to do any Smart Device programming, I highly recommend you uncheck all those packages from the installer. I would also choose not to install Visual J# for obvious reasons. Once the install is complete, be sure to run it at least once. Then install SQL Server 2005 (running as Local System when asked). If you don't care about SQL 2000 support, don't bother with the compatibility sort orders and just pick Latin from the top. Once both are installed, go ahead and install VS2005 Service Pack 1 and SQL Server 2005 Service Pack 2.

At this point, you have a pretty basic system with the needed applications. If you're going to do any work on open source projects, you will need Subversion, TortoiseSVN, and NAnt. Many recommend VisualSVN, but I have never used it so I can't comment but I've heard good things. When downloading applications to install, I recommend pulling them down using Safari into your Downloads folder. Then in your VM, go to the "on my Mac" share and run the installers from your shared folder. I never download anything inside the VM unless I absolutely know it can be trusted (which is pretty much never).

You will notice that I didn't install any "fearware" such as virus or spyware tools. That's my style, I don't really care to slow down my system with things of that nature. I maintain a tight VM and rarely introduce anything from the outside so it's my choice. Your mileage may vary.

And before I write another word, you should install Resharper and start on your path to becoming a Resharper Jedi.

Wrap Up

I hope this post has been helpful for those looking to setup a development VM. I've found that using Leopard OS X for everything but development to be a pleasure compared to the applications available for Windows.

Assert.That(this, Is.Easy);

2008-01-21T16:10:00Z

I came up with this a month or two ago, but finally decided to share it. While working on Mass Transit, I was joking with Dru Sellers about how nice it was to have really good test coverage when making design changes to some all-new development code. I've had very limited opportunity for a completely new projected started purely from unit tests, so I was just impressed at how easy it was to make code changes knowing that a passing set of tests meant all was well in the world.

You see, not all parking lots are paved with quality asphalt, generally flat, and void of any obstructions like islands and lights (see my other hobby). At work, our application is a lot of vintage C++ code, a ton of stored procedures packed to the hilt with domain logic, and nearly zero percent unit test coverage. Since adapting agile development, it is something that has been missing from our process. In our latest iteration, we've started using unit tests (with NUnit) to design our interfaces and classes. At the same time, we're integrating Mass Transit to support the loosely coupled layer of application services (which include object translation, communication with high-latency remote systems, and lazy auditing of transactions). Aside from a few basic web services to support remote client application support tools, this is the first C#/.NET development that is being done as part of the main application.

So back to our story, my first project with really good test coverage exposed me to a lot of new things. From a TDD perspective, I'd read about it, used it to build some basic tests for various classes, and thought I had a pretty decent understanding of it. In this new project, I also learned how to use Rhino.Mocks (which took the test run time from 40-50 seconds down to 1.83 seconds on average), a very powerful tool for making an interface behave as you would expect an implementation of that interface to behave. The use of mocks has really helped me focus on actually writing tests and building a single class at a time. Prior to using mocks I would jump around creating additional classes as I defined new interfaces just to be able to continue writing my unit tests on the original class. By using a mock, I'm able to simulate the behavior of the other class without losing focus.

As my appreciation for TDD grew, I jokingly dropped a slogan into a chat window (using Skype, of course, aren't you?):

I got a few chuckles, and thought it would make a great t-shirt to wear to tech events like code camps. So I threw together a quick online store so that I could order one for myself. I showed it to a few others (like Joe Ocampo, who suggested the slightly less offensive, yet subtly more suggestive variant) and decided to make it available to anyone that wanted one. So if you like it, grab one for yourself and maybe I'll see you wearing it at ALT.NET Seattle!

Resharper 3.1 Solution Wide Analysis

2008-01-06T15:26:39Z

Since early last year, I have been using Resharper to increase my .NET coding productivity. After seeing the product used at a Code Camp, I was intrigued by how it make test driven development so much easier. I started on version 3.0 and JetBrains dropped 3.1 on Christmas Eve so I checked it out over the holiday break.

One of the features that first interested me was the Solution Wide Analysis. I turned it on, looked at the badly drawn green circle in the corner of the Visual Studio 2005 IDE, and went, OK. It took me a bit more time to recognize how this new feature was going to improve my skills yet again.

The first thing I learned was that the blinking light was not the real power behind solution wide analysis. The power was that Resharper was working hard eating up those idle CPU cycles to verify the integrity of my entire solution. It wasn't until I double clicked and put the window as part of my IDE into practice that I saw the true power (of the force?).

The above image shows the errors in solution window with a clean bill of health, no problems with compilation. The next image shows what it looks like when things get broken:

I slipped the window right under my solution explorer, giving me a constant view of my code status. When refactoring, it makes it easy to see all the affected files at a glance. Very cool. But I do have a couple of nags on this feature:

I really don't need a tall bar eating space at the top of the window for a repeat of the red/green indicator -- get rid of it
I would like the option to expand all the nodes in the tree by default instead of only showing the file names

The new solution wide analysis feature is a great red/green indicator to see the impact of code changes. If you haven't upgraded to 3.1 yet, what are you waiting for? And if you aren't using Resharper yet, what is holding you back?

What I Do Is What I Do Is What You Do Or What

2007-12-30T07:40:53Z

Since I got tagged by more than one of my fellow Los Techies bloggers, I figured I better step up and throw down.

While my job history is available on my LinkedIn profile, I'm going to focus on what I do today. The executive summary of the last 20 years spans from Turbo Pascal and dBase III Plus to C# and SQL Server.

For the past six-plus years, I've worked at what is now known as RelayHealth. In various forms (the company, not me), I've worked for the same company since 1993 (except for a 5 year adventure with TV Guide and a startup company). The company provides electronic communication systems for health care providers (mostly hospitals) and insurance companies (payers). Our network is responsible for a majority of all health care insurance transactions within the United States.

Yeah, big words, what but do I do?

Our most recent project (and the reason I returned to the company) was to create a web-based replacement for our legacy client-server claims management system. The system uses a combination of C++, ASP, COM/ATL, SQL Server, and JavaScript to provide an interactive web-based experience for the users. It's a large application, and we have a large team (around 20 developers) working on it. In the past year or so, we've added some new features developed in C# and .NET 2.0, including some web-services for remote integration.

In the past six weeks, we have adapted agile development as our mode of operation. Just this past Friday we completed our second iteration (yeah!) and are preparing to start our next iteration right after the new year. It's been challenging trying to keep up with all the new methods while making sure we continue to meet all of our compliance requirements (SOx, HIPAA, etc.). The holidays really impacted our velocity, as well as our ability to fill up the backlog. Hopefully after the first of the year we'll have better attendance and get some solid plans laid out. For now, a few members of the team are spiked on research for upcoming enhancements.

My toolset includes the following:

Visual Studio 2005, Subversion, Resharper, TortoiseSVN, SQL Server (2000/2005), WatiR, nUnit, CruiseControl.net, FinalBuilder, WISE, PHP, Ruby, and likely some small stuff I left out. Sparx Enterprise Architect on occasion, as well as Mindjet MindManager Pro. We also leverage VMware heavily for development, testing and quality assurance environments. Our application deploys on Windows Server 2003 using IIS6 and SQL Server 2005.

At work, my official title is Software Architect. But I do spend a decent amount of time coding (no non-coding architect here), mostly to determine the feasibility of new technology and then train the rest of the team on how we can apply it to our product.

Outside of work, I spend time with my family, enjoy driving my Subaru STI, and recently added an XBOX 360 for gaming entertainment. I'm a devoted Mac user, rolling on a 15" MacBook Pro for my personal development projects. I also enjoy movies, music, and Guitar Hero (now Rock Band as well). I continue to spend a great deal of time outside of work on open source projects, learning more about my craft (software development), and playing with technology.

I hope to attend (and maybe even present at) several code camps, bar camps, tech fests, or other development meetings in 2008. I feel the software development community as a whole needs to work together (particularly the .NET community) to establish a solid set of practices for building great applications using the best tools available with the least amount of pain and misdirection.

And me, I tag nobody, for I think everyone has been tagged and I'm the slo-poke.