Once you wrap your head around using MVC in your web apps you tend to look at other ways to make your Controllers more descriptive. Specifically the way you structure and document your controllers.

I have noticed that a few portions of MVC design that I can highlight as to where you should pay extra attention and be more discrete in your decisions. These areas are:

• Actions
• View/PropertyBag variables
• Controller/Action reuse

For this posting, I will be using MonoRail in any examples but these ideas are confined to any one MVC framework. You can apply it to your own custom MVC, ASP.NET MVC or anything other MVC framework for that matter.

Actions

Action naming is one area that I think deserves the most attention. There are a couple of different implications of your choices here. You may say, well I can just rename my actions but this drags along some pains because then you have to repoint any hyperlinks you have referring to those actions and refactor any controller code that uses those actions. You can mitigate the hyperlink problem by adding a new Routing Rule to point to the new action, but this should only be used as a temporary fix and shouldn't be a permanent solution.

When naming my actions I always try to limit actions to one word. There are a couple of reasons for this. The less words the easier it is to convey intent of what the action is going to do. This is often hard to convey with one word so sometimes two words are necessary. I try to keep it to one and often cringe when I add that second word. Common action names I have on a controller are, View, List, Search, Find, New, Create so on and so forth. While using jQuery to do ajax calls I am considering prefixing all actions that are called via jQuery with "Ajax". So if I want to perform a Get action that is called via ajax the action name would be AjaxGet, this way it reads like the jQuery methods for performing the calls to a remote page.

Next, I try to steer away from overloading Actions. Sometimes there are exceptions but as a rule of thumb, if I am overloading my actions I take a second look at why I am needing to do this and usually gives me a hint that I need to refactor some code on the controller like break it up into two seperate controllers. This is often the case.

Action names should always be performing that verb on the controller at hand. So for a Controller named UserController, any action should "usually" be working with a User. Some actions do turn out to be utility actions however so use your judgement as to when you can stray away from this idea.

View/PropertyBag variables

When I refer to "View/PropertyBag variables", I am referring to data that is sent to the view from the Controller so we are all on the same page.

On naming of your ProperyBag variables, I always am careful to be consistent with my pluralizing here. If I am sending a collection of objects, be sure to make the variable plural so that you don't mix yourself up. Other than that, I just remain consistent with entity/dto naming in my domain so that it is consistent across the board.

Controller/Action Reuse

This is one area that is a struggle. The easiest and most commonly used approach is Controller inheritance which works without a hitch. In my experience this works well for small scenarios with minimal reuse. Once you get into more complex Actions it becomes cumbersome and undesirable to attempt to apply inheritance to your Controllers. There are a couple of things that can help once you outgrow inheritance.

The first is MonoRails DynamicActions. DynamicActions are a way to create an action in it's own class and then by using attributes, you can hook up the disconnected action to whichever Controllers you wish, passing in parameters that deal with the specific context the DynamicAction is being used in. This works extremely well if the code that needs to be reused is exactly the same except for the objects that it is working with. For instance, list and view actions.

One area where I have found Controller inheritance to work very well is in the area of managing security in your Controllers. What I commonly do is make a "SecureControllerBase" that has an Authentication Filter applied to check if the current user context is authenticated or not, redirecting where appropriate. I commonly use these base controllers for applying filters/monorail attributes across all controllers. That comes in pretty handy.

That concludes my little blurb on Controller design/layout. If you have something to add please add a comment.

Due to a spell of blogger's block, I thought I would do a series postings to help me get past it. This post begins a series on highlighting various features of the Castle Project MonoRail. All series will be examples from the MonoRail trunk. Some of the features I will be discussing are only available on the trunk (specifically the new Routing Module). To start off the series. The first part will cover MonoRail helpers. The series will have the following topics:

   1:  $FormHelper.TextField("blogpost.title")

   1:  <input type="text" id="blogpost_title" name="blogpost.title" />

What happens here is the FormHelper creates the requested form field and sets the id/name attributes properly so the data can be data bound on the controller action which would look something like this:

   1:  public void CreateBlogPost([DataBind("blogpost")] BlogPost blogpost) {}

   1:  public class MyCoolHelper

   2:  {

   3:      public string DoSomethingCool()

   4:      {

   5:          return "cool stuff";

   6:      }

   7:  }

   1:  [Helper(typeof(MyCoolHelper))]

   2:  public class SomeController

   3:  {

   4:  }

Next time we will go over layouts, views and view components. Should be a fun time!

Writing Domain models is by far one of the most difficult things to do. It takes years of practice, errors and learning to get good at writing halfway decent domain models. This becomes even more difficult when we attempt to add ideas like Domain Driven Design to the mix. I am not proposing I am even remotely good at it, but I have had lots of practice.

In the last two weeks my co-worker and I have begun a greenfield project. This project is a complete language re-write due to the fact that the existing technology it is written in, is a dead end. There is no upgrade path and the staff running the software have become more ancy as time has progressed because they knew sooner or later they would have to leave the existing platform behind. The project is greenfield because of no technology constraints, that doesn't mean however there isn't domain language constraints.

The problem with this project is only a handful of people are intimately familiar with the domain. Even then, the few people that were familiar don't agree often on how specific aspects should work. This adds an enormous amount of risk to the project as even though there is a functioning domain model, it is one without a solid foundation and thus weak model.

Enter Domain Model Refactoring...

A few words before I begin. This is an extremely risky thing to do when using Agile/DDD concepts. You run the risk of alienating the client from the domain language. Not only are you going out on a limb here, but you are playing with the business language that the client is using. If I wasn't comfortable with the client enough to talk to them on a real level (my client is within my staff), then I wouldn't attempt to refactor the model for their benefit. Trust is extremely important between the client and developers, as it normally is, but more than usual. Couple that with the fact that in my instance there is no money changing hands, so there is little or no monetary cost involved, that doesn't mean however that there isn't cost in general involved. Cost in this sense is time and effort, not money, although in some instances, time and effort is proportional to money but not in my case.

Usually you are working with a project where you are starting from day one and can work with the client on the domain model. In this instance, they have been using a model for years and I am attempting to change it to make it more effecient for them. Again, proceed with caution.

Step 1. Model the existing Domain 

The first step we took is to model out the existing solution with aggregates/entities and value objects. This gives us a good idea of how the model "would" be modeled if it was in a pure DDD type model with all the bells and whistles. This allowed us to reveal holes and gaps in the model (and there was a lot of them). When we approached the client about this, they shrugged and claimed they didn't know why functionality was like it was, just that it was that way. This allowed us to refactor parts of the model that were superfluous and began to make the model more lean.

The existing domain was modeled after the technology it was based on. Because of performance constraints, specific things had to be done in a specific manner in order to salvage performance. After noticing this in the DDD model we created, we refactored the "performance features" into more logical operations with extra documentation. The original application was split into seperate databases that were converged on a daily basis in the morning. Because of this, each database was it's own seperate bubble of transactions/processing. To minimize collisions when everything was merged together, the model was setup in such a way that extra processing was required on the actual users of the system to resolve these conflicts before they happened. We were able to again make the model more lean by removing these constraints as we were refactoring to a single database approach and could remove these constraints.

Step 2. Refactor to a more logical model 

Once we had this step complete we could then begin to refactor the model into a more logical manner. This should be done with baby steps and each change be communicated to the client. As long as you have a good idea of what the model should be, then make appropriate changes to the model. All the while documenting what has been refactored into what. You will need to communicate these changes to the client, otherwise you will be the one translating the old model to the new model. This is to be expected at first, but once the client gets running on their own feet they should be able to do this themselves, if they are not then you need to provide more documentation and explanation as to why a particular portion of the model is setup a specific way. In this case, the client didn't actually care what language we changed as long as it still served their purpose and we informed them of any definition changes and provided documentation on why a specific feature was dropped.

Before we laid down a single line of code, we spoke with the client explaining exactly what was happening, why it was happening and then received feedback from them. This part is very important as it will uncover MORE gaps in the model as well as gaps in their understanding of the model. Have someone take notes and address the gaps accordingly in your refactored model. We did this step and realized that some of the changes we made didn't need to be communicated to the client because they already took this for granted, however, we wouldn't have known this unless we spoke with them about it.

Looking back in this project, the key was communication and feedback. If you cut this portion out of the loop you increase risk very quickly. Don't take anything for granted and leave out assumptions. It is better to ask 10 questions and get 9 dumb answers and one good one than to ask no questions at all.

As I said in the beginning of the post, I am no expert on modeling but I have found that this approach has worked well for me and my team. Hopefully you can take something from it.

In an attempt to spread awareness in the North/Central Florida area on Alt.Net topics I am trying to gather speakers familiar with Alt.Net topics to speak at the Jacksonville Code Camp that is coming up on August 23rd. The topics I am envisioning are Domain Driven Design, TDD, BDD, NHibernate, ORM's in general or any other topics that would be of interest in the alt.net space.

If you live in the central/north florida area and are interested in speaking at the code camp please register at http://www.jaxcodecamp.com and please contact me at dkode8 at gmail dot com. This way I can begin to organize an alt.net track at the event when I attend one of the planning meetings.

Hoping to hear from you!

Late night thoughts after a coding session before going to sleep. 

One skill that I value above all others in software development and life in general is the inner drive and motivation to do better and to do more. It is a skill that can be honed over time. Once you begin to cultivate it, a snowball ensues getting faster and larger as time passes. I am immensely more motivated and driven now than I was a year ago, and the year before that. This is from reading books, writing code, doing presentations at code camps and user groups, getting involved and giving back the community as much as I take out. As well as pushing myself to learn the next step, and the step after that.

For me, it feels almost like a struggle, if I feel myself slowing down or reaching the pinnacle of a particular topic of knowledge, I find another one to begin learning and transition. It's a zone in between comfort and frustration. It's very hard to describe and must be experienced by the inividual doing the exploring. Too much at once causes frustration, not enough precipitates laziness and comfort. There's a balance in between that can be tailored and tuned to.

This applies to all aspects of life, Continuous learning and improvement must always be on your mind. If you stagnate, then you go the way of the dinosaur. Especially in the software industry.

This concept can go from extremes from the macrocosm of your life, all the way to each individual line of code you write. Everytime you look at piece of your own code you should consider that it could possibly be written better at the same time you should be looking at your life and how you can be better to your surrounding environment and fellow beings.

 I may be getting a little too zen there however =)  Just a thought!

Over the last couple of months there have been a slew of great refactorings done on the castle monorail trunk. Some of these refactorings were inspired by ASP.Net MVC changes such as the new routing module. I have just now started to play with monorail trunk so I am by no means an expert on the changes that were done there. If you are interested in getting up to speed as well. The best thing to do is get a fresh copy of the monorail trunk and start looking through the source code specifically on Controller, EngineContext, ControllerContext and IController. After looking at the changes for only an hour or so and updating a project up to the monorail trunk, I can already get a good idea of where things logically should belong now.

General Changes

• The controller class has been split up into Controller and ControllerContext. A good deal of Controller "metadata" has been moved to ControllerContext. In addition, an IController interface has been created that is now used within all the Castle services.
• RailsEngineContext has been renamed to simply EngineContext.
• The ExecuteEnum has been renamed to ExecuteWhen which makes more sense to me.
• Certain services have been removed from the Controller
• The EngineContextModule has ceased to exist

Routing Functionality

This is the coolest part of the refactorings. Instead of relying on ugly xml configuration for our routes, we can now configure them within our HttpApplication class using the PatternRoute class. This was the hardest portion to find documentation on. There are some postings on the castle project google group that helped me along. You can find the posting here. In addition to reading that, the best way to get familiar with the new routing functionality is to download the trunk and take a look at the Routing tests. They are pretty descriptive and allowed me to wrap my head around them. To outline some of the routing functionality I have an example of a route in one of my applications:

   1: rules.Add(new PatternRoute("/<controller>/<action>")

   2:     .DefaultForAction().Is("index"));

The rules class is an instance of RoutingModuleEx.Engine that you can obtain in your global application. We then pass the Add method a new PatternRoute. This is where we define the route to match.

This route will match anything followed by anything, It will then map anything between <controller>, and pass it along as the controller parameter, and pass along <action> as the action parameter. If no action is passed, a default of "index" will be applied. Specifying a parameter as [something] makes it optional, while <something> is a required parameter. The thing that took me a minute to understand is, when constructing the route, anything you make as a parameter is passed along to the action at hand. This means that if you have a parameter to your action that is someThing, you can place it in your route and that part of the url will be passed along as that parameter. To display this I have another example:

   1: rules.Add(new PatternRoute("/something/<parent>/<param2>/<param3>/")

   2:     .DefaultForController().Is("somecontroller")

   3:     .DefaultForAction().Is("view"));

This route is a little more complex. To get a better idea of how this works, take a look at the associated action signature:

   1: public void View(string parent, string param2, string param3);

As you can see from this example, you can name the parameters to pass along to the action. On that same note, you can specify parameters that aren't included in the url like so:

   1: rules.Add(new PatternRoute("/area/<param1>")

   2:     .DefaultForController().Is("someOthercontroller")

   3:     .DefaultForAction().Is("view")

   4:     .DefaultFor("someOtherParam").Is("someValue"));

In this example we are calling the view action on SomeOtherController that would take a signature as follows:

   1: public void View(string param1, string someOtherParam);

As you can see, the someOtherParam will always have a static value that we define. Not sure if this would be useful to most people but I have already had a use for it when constructing dynamic routes.

One of the more complex parts of the refactorings is definately the routing. There are quite a few posts on them on the Castle Project Users Group and the Development Group so do some searches there if you have more questions about the routes.

That's it for now. Next time we can dive more into the refactorings and changes on the trunk. Feel free to ask any questions!

FYI, Just read this posting, good news! Resharper 4 Release Candidate is now available. The last time I udpated from the nightly build has been pretty stable. A couple of minor bugs but no biggies.

 Good job guys!

Database Refactoring is always a pain to be dealt with in any medium-large sized software. There are new tables, columns, primary keys, foreign keys added and removed constantly from a database under development. Some people have a strict plan of creating sql scripts for every change they make in the database. Others have a text file holding a listing of changes to be done on QA and Production machines along with migrating existing data into new structures. Anyway you slice it, It's a headache all around and if you ignore it, it will eventually bite you in ass.

This is one area that they Ruby crowd has gotten in right (not saying that they haven't got other things right), but with Migrations, it is amazingly easy to create objects in your database and then easily version your database. Originally an open source utility was written by Marc-Andre Cournoyer called .Net Migrations (or something to that extent), it lived in the castle trunk for awhile but once Marc-Andre went to the ruby crowd Nick Hems picked up the project, dusted it off and placed it on Google Code located here.

Ok, now that we have gotten the history lesson out of the way let's get started. Earlier this evening the trunk I updated with a major set of refactorings done mostly by Geoff Lane, which we have been toying with for about a month or so. Now that Geoff and Nick fixed a number of bugs, we updated the trunk so everyone can have fun!

One of the best parts of this database refactoring tool is the database support. At the current moment, Migrator.Net supports MsSql Server, MySql, Oracle, PostgreSql and SQLite. You need only specify the connection string for the database and the provider to use in an NAnt task that was developed for Migrator.Net like so:

<migrate provider="MySql" connectionstring="Database=dbname;Data Source=localhost;User Id=username;Password=password;" migrations="Project.Migrations.dll" to="2" />

Specifying a "to" attribute of 2, will migrate your database to the next available version. When you first run migration version 1, a "schemainfo" table is created in the database that keeps track of the current migration version. Then by specifying a 2, the next version is applied to the database.

Where do I create my migrations you ask? Good question!

I create a seperate assembly, in the nant task above it is called Project.Migrations.dll. The migration looks something like so:

   1: // Version 1

   2: [Migration(1)]

   3: public class CreateUserTable : Migration

   4: {

   5:     public void Up()

   6:     {

   7:         Database.CreateTable("User",

   8:             new Column("UserId", DbType.Int32, ColumnProperties.PrimaryKeyWithIdentity),

   9:             new Column("Username", DbType.String, 25)

  10:             );

  11:     }

  12:     public void Down()

  13:     {

  14:         Database.RemoveTable("User");

  15:     }

  16: }

Ok, so it doesn't get much easier than this. It has a feel exactly like Migrations for Ruby, so anyone familiar with that should have no problem with this. The first thing to take note of is the Migration attribute with a 1. This denotes what version this migration is. Each time you create a new migration, it has to be the next available number. This is required. Next is the class name which can be whatever you like for readability purposes. Next is the Up() and Down() methods. When migrating TO this version, the code in Up() will be called. When rolling back a change to a previous version, the Down() method will be called. Easy enough.

If you take a look at the Database class, you'll see that there are many other methods there for creating ForeignKeys, PrimaryKeys as well as other column information to be applied in the database. Column options can also be passed in the Column() ctor overloads.

The project can be found here: http://code.google.com/p/migratordotnet/

The google group where I make silly postings is here: http://groups.google.com/group/migratordotnet-devel

The next thing I think we'll probably look at is refactoring towards a fluent interface as this is clearly a fit for that type of implementation. Go download it and take a look! Please post any bugs to the tracking list on google code.

Next time I'll go over rolling back changes and other more complex changes to perform on your database. Enjoy!

Just came across a situation where I needed to have a collection of enum values mapped to an IList and to have it supported by NHibernate. It took me a little bit to find the proper approach and get NHibernate to play nicely. So as a reference for anyone else running into this you can find some information at this posting here

Here is a snippet of the bag at hand:

<bag name="companyRoleList" access="field" lazy="false" cascade="none" table="CompanyRole" >
    <key column="CompanyID" />
    <element column="RoleID" type="MyCompany.Domain.Lookups.CompanyRoleType, MyCompany.Domain.Lookups" />
</bag>

This is from the blog post on the NHibernate forum. The trick here is to have the full namespace/assembly qualification in the type attribute that points to the Enum you are using as a collection. This is needed even if you have the namespace=, assembly= at the hibernate-mapping element. That stumped for a little while.

Hopefully this helps someone else out!

Even though I haven't been blogging alot recently on LosTechies, doesn't mean I haven't been blogging. Even though Bil Simser, Sergio Pereira and Kyle Baley have all claimed to be ALT.NET Pursefight, they are indeed NOT!

I am the true ALT.NET Pursefight!!!! Accept no substitute and believe noone else!

Who else could it be? I am witty, intelligent and cool all wrapped up into one. This clearly makes me the ALT.NET Pursefight blogger. I am one of the "nobody developers" that would clearly end up being the ALT.NET Pursefight blogger. It's as clear as mud!

At first I tried to chime in once or twice on the list only to get burned. This make me quite angry so rather then lash out I decided to don a cloak of anonymity and attack from behind the shadows. Genius eh? Before you ask I WILL NOT sign any autographs or kiss any babies.

That being said, the list has become quite boring recently so I urge everyone to start bickering again. Give me more blogging material! PLEASE!!!

I originally heard about the storm worm/botnet about 6 months ago but wrote it off as another b.s. botnet used by spammers for mediocre spreading of email. Recently I saw some newer articles and started looking into it again. This is when I realized how diabolical, beautiful and well crafted of a worm this beast actually is.

Originally discovered in early 2007, the storm worm was thought to just be another worm that spread randomly infecting machines via e-mail. Upon further investigation, experts realized that this worm was something much more dangerous than a basic worm infecting machines. The storm worm uses an exploit in Windows XP to propogate to machines. No surprise there.

The storm worm is actually multiple different programs rolled into one. It is actually a massive well crafted botnet consisting of anywhere from 160,000 to 50 million machines total. Recently, security experts have developed spiders to crawl the botnet and place the estimate more towards 160,000 computers rather than several million. Along with worm propogation the botnet also performs DDoS attacks, spamming, command and control servers, an e-mail address stealer along with many other duties.

An interesting attribute that the botnet possesses is resistance to probing and inspection, almost like a defensive barrier that reacts to any outside intervention. Upon scanning and/or crawling the botnet, experts found that the botnet instantly recognizes that someone is trying to inspect it and retaliates with a DDoS attack. Whether this is an automated process or is being controlled by a user is unknown, although experts assume that it is the former. A very innovative feature indeed.

With enough processing power to rival most of the worlds supercomputers it is a scary idea that such a resource is being controlled by criminal intent. When the botnet does commit a DDoS attack, it contains enough firepower to take an entire country offline. To quote wikipedia, "The webmaster of Artists Against 419 said that the website's server succumbed after the attack increased to over 400 gigabits per hour of data, the equivalent of over 170,000 ADSL-connected machines". This is an amazing amount of bandwidth and a scary prospect indeed.

The botnet operates on a modified version of eDonkey's peer-to-peer networking which makes it almost impossible to take the botnet offline. Just like file-sharing p2p networks, no matter where you attempt to disassemble the network there are always other machines to take the place of patched ones.

Only specific portions of the botnet are dedicated to certain tasks. A small portion provides DDoS attacks, another portion is strictly for spreading the worm to other machines while an even smaller portion is used as command centers to spread commands to the dormant machines.

So you ask why can't you just find out where the commands and updated virus packages are being sent from? Good question. The controllers of the botnet are using a constantly changing DNS technique called "fast-flux", this allows the host machines to be almost impossible to find. Couple that with the fact that almost all communication within the botnet is encrypted and one can see the difficultly in analyzing such a beast.

There are speculations that the size of the botnet has recently decreased but it still remains a very real threat to the internet as a whole. Especially being in the hands of criminals. I am considering setting up a dummy box to purposely get infected to "play" with the worm so to speak. Although, this could very well mean the death of my cable connection =)

After Chad and Ray I followed suit as well and am doing Pablo's Topic of the month post on the Single Responsibility Principle or SRP for short.

In SRP a reason to change is defined as a responsibility, therefore SRP states, "An object should have only one reason to change". If an object has more than one reason to change then it has more than one responsilibity and is in violation of SRP. An object should have one and only one reason to change.

Let's look at an example. In the example below I have a BankAccount class that has a couple of methods:

   1: public abstract class BankAccount

   2: {

   3:     double Balance { get; }

   4:     void Deposit(double amount) {}

   5:     void Withdraw(double amount) {}

   6:     void AddInterest(double amount) {}

   7:     void Transfer(double amount, IBankAccount toAccount) {}

   8: }

Let's say that we use this BankAccount class for a persons Checking and Savings account. That would cause this class to have more than two reasons to change. This is because Checking accounts do not have interest added to them and only Savings accounts have interest added to them on a monthly basis or however the bank calculates it.

Some people may say that the class would even have 3 reasons to change because of the Deposit/Withdraw methods as well but I think you can definately get a little crazy with SRP. That being said, I believe it just depends on the context.

So, let's refactor this to be more SRP friendly.

   1: public abstract class BankAccount

   2: {

   3:     double Balance { get; }

   4:     void Deposit(double amount);

   5:     void Withdraw(double amount);    

   6:     void Transfer(double amount, IBankAccount toAccount);

   7: }

   8:  

   9: public class CheckingAccount : BankAccount

  10: {

  11: }

  12:  

  13: public class SavingsAccount : BankAccount

  14: {

  15:     public void AddInterest(double amount);

  16: }

So what we have done is simply create an abstract class out of BankAccount and then created a concrete CheckingAccount and SavingsAccount classes so that we can isolate the methods that are causing more than one reason to change.

When you actually think about it, every single class in the .Net Framework is violating SRP all of the time. The GetHashCode() and ToString() methods are causing more than one reason to change, although you could say that these methods are exempt because they exist in the framework itself and out of our reach for change.

I'm sure you can come up with a lot more instances where you have violated SRP, and even instances where it just depends on the context. As stated on Object Mentor: "The SRP is one of the simplest of the principle, and one of the hardest to get right".

Here is a link to the SRP pdf on Object Mentor for more information.

I will be doing a presentation at the Orlando Code Camp on next Saturday on March 22nd on Continuous Integration with TeamCity.

I will be mainly cover the topic of Continuous Integration and how teams can benefit from it, then I will show how TeamCity works and dive into some of the details with this great tool. JetBrains just released 3.1 which improved TeamCity in various ways.

If you are in the Orlando area you should definately try to get to the code camp. There is a lot of great topics lined up.

Hope to see you there!

About a year ago I setup a Ubuntu machine on an old P2 400 at home to use as a subversion repository. At the time I created one repository to house all of my projects. Over time I was adding more and more projects to it all under one master repository path. This was fine when there was 2 or 3 projects, but after growing to about 15 different projects, performing selective dumps and repository maintenance in general becomes a little cumbersome. I recently decided to break out the large repositories into their own repositories under the same subversion path (/var/svn/project). The original large repository was located at (/var/svn/repos).

The idea here is that we will first dump the entire repository, and then pipe the output of the dump command to svndumpfilter which will extract only the path that we want. These two commands together is all that is needed to perform selective filtering of paths within a subversion repository and then dump the resulting output to the project.dump file. Unfortunately the svndumpfilter can only take either include or exclude in a command but not both, this limits it's usage in advanced scenarios, although it will work for what we are wanting to do here. Easy enough.

With some piping we can perform all of this in one step:

sean@svnbox:/var/svn$ sudo sh -c 'sudo svnadmin dump /var/svn/repos | svndumpfilter include projectfolder > project.dump'

The reason for the sudo sh -c '' part, is so the piping of output of dump can be directed to master.dump with superuser permissions. In ubuntu everything is done with sudo instead of switching to root using su, therefore the first part of the command is evaluated using sudo, but the output being piped is evaluated using privileges of the shell. To make a long story short, if you are using ubuntu, you need to execute the command in it's own context thus, wrapping the command in it's own shell. If you were using any other distribution you could first switch to a root shell using su, then do the above command with the sh part.

Now we can create the new repository, and load our filtered dumpfile into the new repository

sean@svnbox:/var/svn$ sudo svnadmin create /var/svn/project | sudo svnadmin load project < project.dump

The only thing that stinks now, is that since the single repository had a folder for each project, the project.dump now has a single folder in the root that is named project. To fix this, you just need to move your branches,tags and trunk folders into the root manually using svn mv or with tortoisesvn.

In my case, I also had to edit /etc/apache2/mods-enabled/dav_svn.conf in order to accomodate the seperate repositories by adding the following:

# from /etc/apache2/mods-enabled/dav_svn.conf

# Remove/Comment out SVNPath
# SVNPath /var/svn/repos

# Added SVNParentPath
SVNParentPath /var/svn

SVNParentPath tells apache that there are multiple repositories hosted under /var/svn and that should be considered the root of your <Location> tag in the dav_svn.conf file. Clear as mud?

In the future I will definately think twice about creating everything into one master repository, at the time I was just learning subversion and got started the quick and dirty way. In some cases however, it may make sense to have everything under one repository rather then break it out seperately.

Hopefully this points someone in the right direction. Most of the above is documented very well in the SVN Book, so finding documentation on the above procedures and svndumpfilter should be easy.

For awhile now I have been wrestling with SSIS. I don't like it at all so much to the point that the majority of my nightly packages are still on SQL 2000 as I have been unsuccessful in porting them to SQL 2005 SSIS.

A little while ago I went searching for alternatives. There are quite a few out there, some having very good reviews. I don't have a list of what ones I looked at as this was a couple of months ago. The one that did stand out was Pentaho. Not only does it do everything and more than SSIS, but specific verions are available as open source. Namely, the Pentaho Data Integration is available as open source. This is the main ETL tool that they have and is written in Java. It is very mature and has a wealth of documentation that accompanies it. This was the deciding factor for me.

Originally I was using RhinoETL, but the only problem was the lack of documentation. This was ok for simple tasks, but once I had to do more complex data processes, and my lack of knowledge with boo it became more difficult to stay with RhinoETL. Ayende has a great tool, it just needs to simmer for a little while I think. I would definately consider going back once it has matured a little more.

In addition to the great documentation, Pentaho Data Integration also has a visual designer for creating data workflows. Here is one screen shot of a simple process that exports to a csv, uploads to a remote FTP and then send's an e-mail upon success or failure of the task.You can see a few of the tasks available to you on the left sidebar.

Everything in Pentaho is in a propietary Package Repository on the server so if you mess up an export you can rollback the repository to previous versions. Very nice feature!

Here are some more screenshots of the interface:

Very basic export with email

E-Mail configuration window:

Options for attaching logs to the e-mail to send:

This is just barely scratching the surface of the processes you can perform with this tool. The Data Integration server and clients are all available under open source liscences. As well as access to the forums and all the accompying documentation.

I will post more on the topic later. It has just been a very helpful tool and I think other people need to see some alternatives to SSIS.