Wednesday, April 23, 2014

Introducing the new Portland F# Meetup Group!

I'm very excited to announce that I, with the help of Mathias Brandewinder, am launching the F# meetup group for Portland. He has agreed to come up and spend some time here with the local .NET community and lead a couple of workshops. The first one will be an F# for C# developers talk through the Portland Area .NET Users Group (PADNUG) and the second will be to kick off our first F# meetup.

You can read more about the F# event here and, please, sign up and help get the word out. I am excited about the potential of getting this group running. I hope to have programmers of all skill level and experience come out.

Our first meetup will be a three hour, hands-on, dojo going over machine learning to recognize hand-written characters. If that sounds at all interesting, even if you're not an F# develop, please come and check it out. Come for the machine learning, stay for the F#! Bring your laptops (or share with a friend) and let's get our fingers dirty on some code.

As I get a feel for how many people will be showing up, we will also be looking for a venue. If you have one and are willing to host us, please let me know! We will need some wifi and a projector and that's about it.

I look forward to seeing you there!


We now have a venue setup! It is located at the Beaverton Community Center (12350 SW 5th Street, Suite 100) in the Vose room.

Sunday, March 23, 2014

Quick and Dirty Command Handlers, Event Handlers, Wiring it Up

In my last post, I talked about a "quick and dirty" way to create a command bus in F#. In this post, we're going to look at a simple set of command handlers to plug into said bus and some event handlers.

Before I jump right in, let's define what we're looking for. I want to use NEventStore for my event bus (comes with some nice features that make getting up and running much faster). Feel free to plug in whatever event store you might feel like. Creating (an extremely basic) event store isn't too terribly hard, but I am not going to cover it in this post. So, without dragging this out too much more, let's create a function that wraps up the creation of NEventStore.


NEventStore needs a serializer. It comes with a few baked in, but I have grown fond of FsPickler. It is really fast and convenient since we're using F#. Here's a quick little helper to create our serializer on demand.


NEventStore also accepts a dispatcher (the thing that is given the events as they come in). Here is where we're going to accept (for now) our event handlers. If needed, as the documentation suggests, we could fire these events off onto a bus like NServiceBus or Mass Transit. All we're going to do (again, for now) is create a function that will return a dispatcher that passes each event to our event handlers. An event handler is just a function that takes a Guid (the entity's id) and a list (ResizeArray) of events.

Create the Event Store

And, finally, we need to create the event store. There is nothing fancy here, it is pretty much just like the example documentation, just using our serializer and dispatcher. It will also use SQL Server as the backing store along with storing the streams in memory.

So, now that we have an event stream, we can push it into our command handlers and extend our setup function from the last post. We're going to change our command handler function to take in bus. We can simply map the original functions and "apply" the event bus, and then pass the resulting functions into the original "createCommandRouter" function.

A simple little domain

Let's take a look at a silly example of a domain, a counter. The counter can be incremented and decremented and that's about it. Since this is all in F#, we'll use a record to define an entity's state and discriminated union to describe the commands and events. This is a pretty common theme if you go hunting around the internet and it has worked out very well for my projects. Our entities will consist of a state, an apply function, and a handle function. (Lev Gorodinski has a really good write-up on this approach and was the basis for my personal "Ah-ha!" moment.)

Now we just need to wire-up our command handler to use the apply and handle functions. Please keep in mind that this is a "Quick and Dirty" solution to get us going and exploring our domain and wiring things up. You will likely need to a function that the handler can call if it encounters and exception and will probably need to rework the domain to return multiple events from commands and then make the appropriate changes in the command handler to handle that. (Simple, really, but too much noise for this post. I will explore these things in a later post.)

Event handlers are just as simple. They can be used to respond to events from your domain and do things like update projections that feed the application. Our "q 'n d" event handler looks something like this:

The DomainStartup

Putting this together, we now get our new domain startup.

This post is already getting long so I will end it here. Have any feedback or questions, please leave them in the comments or hit me up on Twitter. I'd love to hear how you get started quickly on projects to feel productive without building too much cruft.

Saturday, March 8, 2014

Combining F# and Simple.Web to Break SqlEntityConnection

Before we begin, I want to make it absolutely clear that I believe the problem I ran into this week can be attributed, 100%, to PEBKAC.

So, I am creating a new application that is supposed to "enhance" (replace) an internal application and I want to show off how productive F# has been lately. One of the coolest things I've seen with F# are type providers. To connect to the legacy data in this application, I used the SqlEntityConnection and everything was running along smoothly. (I was basically using the type provider as a read-only view of the legacy data, with a few writes here and there.) I ran into a really weird issue that caused me to lose a little bit of time and just want to document it here so I could maybe save future me (or future you) some headache.

I've pushed up a Git repo with a solution that can be run to demonstrate this issue. So, here we go.

The Symptoms

So, I had about 25 Simple.Web handlers firing on all cylinders and, suddenly, one handler would not work. It continued to return a 500 and I couldn't get a breakpoint to catch. Well, I couldn't even get a breakpoint to set. Every one I'd put in gave me an issue saying that symbols were not loaded for the assembly. No matter how much I did "Rebuild All" it would not change. I finally noticed a weird error that looks like this:

A first chance exception of type 'System.NotSupportedException' occurred in FSharp.Core.dll
System.NotSupportedException: LINQ to Entities does not recognize the method 'BlowingUpSqlEntityConnection.Web.Index CheckThis[Index](BlowingUpSqlEntityConnection.Web.Index)' method, and this method cannot be translated into a store expression.
   at Microsoft.FSharp.Linq.QueryModule.CallGenericStaticMethod@337.Invoke(Tuple`2 tupledArg)
   at Microsoft.FSharp.Linq.QueryModule.clo@1741-1.Microsoft-FSharp-Linq-ForwardDeclarations-IQueryMethods-Execute[a,b](FSharpExpr`1 )
   at BlowingUpSqlEntityConnection.Web.Index.getEntity() in C:\Users\James\documents\visual studio 2013\Projects\BlowingUpSqlEntityConnection.Web\BlowingUpSqlEntityConnection.Web\Home.fs:line 15
   at BlowingUpSqlEntityConnection.Web.Index.Simple-Web-IGet-Get() in C:\Users\James\documents\visual studio 2013\Projects\BlowingUpSqlEntityConnection.Web\BlowingUpSqlEntityConnection.Web\Home.fs:line 29
A first chance exception of type 'System.NullReferenceException' occurred in BlowingUpSqlEntityConnection.Web.dll

It's like Entity Framework is trying to run a stored procedure that matches my handler. Weird. Here's what the handler looks like (taken from the available repo).

With a simple change everything worked just fine. By pulling out the Id and making it an parameter of the getEntity function everything went swimmingly.

And now the application is happy as a clam.

Tuesday, February 25, 2014

Quick and Dirty Command Bus with F#

When developing a new application that could benefit from the Domain Drive Design (DDD), Command Query Responsibility Segregation (CQRS), and Event Sourcing (ES) I like to start out with a "quick and dirty" stack, if you will. It should meet the needs of the application in the beginning with extensibility points baked in so that I can scale out in the areas that make sense. One of the hardest parts about this is worrying about how to serialize commands to command handlers in a way that can be scaled.

One of the interesting pieces of F# (which is not F# specific, of course) is the built-in use of agent-based programming. Using the built-in MailboxProcessor we can easily write a miniature command and event bus. Not having to worry about how to build this specific piece of infrastructure (to start) allows us to focus on building the application and then plug in new pieces as we need. For many applications this approach will probably be "good enough" and might solve our needs.

So, let's take a look at what we want from our buses. Let's ignore, for now, the fact that a command should only every be handled by one command handler. Our application is just starting out, it's small, and we can keep the pieces in our head. We aren't loading up assemblies in some dynamic nature so we can visibly look at a module and see what handlers are registered. (If you can't do this, it might be time to move up a level.) Since we're not going to explicitly determine that a command must only be handled by one handler, we can assume that the two buses (Event and Command, that is) can behave the same way. A "message" comes in and it gets handed off, in a serialized fashion, to the "listeners".

This post will cover the command bus.

Command Router Interface

Let's define a little interface for our command router (the bus).

This simple little thing allows us to create different buses as we need. Of course, to start, we'll just need something to allow us to send commands into the domain. Let's now define the look of our command. We probably want to know who sent in the command, the id of the entity we're targeting, and the command itself.

So here we have a discriminated union that describes who sent the command, a record that describes some "header data" that we might need, and then the "command." The "body" portion of the "Command" record is where we'll tuck away the actual command. (We're using obj instead of a generic to make our life a little easier later. Cheating, yeah.)

Create the darn thing

Let's put our IRouter interface to work. We're only going to have one instance of the bus but we might have multiple buses. We can put together a helper function that takes in our "handlers" and returns us an instance of the bus. A handler is just a function that takes a command and returns unit (void).

Now we can wire it up to our application. Since I am partial to Simple.Web and StructureMap, I'm going to show that here, but it should be fairly obvious how to do this in your preferred framework.

I'm punting for now and putting in some fake "createHandlerN" functions into a list and then passing it into the command router function. In my next post I'll cover what those look like and how to make this play nicely with a simple event bus using NEventStore. Until then, I'm always open for feedback on what I have presented and am interested in how you might have solved this sort of problem yourself! Overall, though, nothing difficult about this and it will help us break apart our applications very nicely.

Sunday, February 23, 2014

Simple.Web F# Project IPathUtility Fix

I just pushed an update the F# Simple.Web Visual Studio extension with a minor (well, pretty major) update. When I removed the Simple.Web.AspNet NuGet package, it also removed a utility that is required by Simple.Web (and is the only real code baked into Simple.Web.AspNet). Since Simple.Web is not reliant upon IIS and is written on top of OWIN, it requires an IPathUtility instance to know where to load the files. The interface is very simple and is found in System.Web.Hosting. I basically lifted the code out of the NuGet package and added it to OwinAppSetup.fs. That way, if you don't want to use IIS, you can easily change the code right there to map virtual paths to the hard disk however you want.

Our OwinAppSetup.fs now contains:

I'm sorry if this caused any headache for anybody!

Thursday, February 6, 2014

Please Don't be Afraid of Command Query Responsibility Segregation (CQRS)

I know that I'm not the only person who has said this and I'm probably not going to be the last, but after discussing this very thing with a few people I think it is worth emphasizing. When I talk to other developers about what images the four words "Command Query Responsibility Separation" conjure, it is interesting. Often times I hear elaborate stories about event sourcing, eventual consistency, and a lot of crazy technology stacks.

It is a shame that this idea seems to be so prevalent, causing a lot of developer to discard it as overly complicated. If we look at the words that comprise the acronym, it seems silly that we attribute so much to it. In my mind, we have wrapped up so many ideas into this simple concept that it is almost like freaking out because somebody is attempting to use SOLID development practices. (Watch out for all of those crazy classes and interfaces!) At its core, CQRS is simple. Very simple. The idea is that we recognize that a call from an outside entity to mutate the state of our application and a call to get state information are two totally separate things. And, as such, should be modeled differently.

Real world pain

I worked on a project that interfaced with a COBOL "backend" for the government. One of the things that we had to face on that project was how the COBOL programmers viewed the world. When we needed to query anything or send in a "command" it would use, in their terms, the same screen. Basically, they had an old green screen AS-400 set of screens (over 200) that we just pushed data into and pulled it out of over a WSDL endpoint. It made it absolutely impossible for us to gain any context on the web side of the world because the same "screen" was used to ask for data and to mutate it. We also had to serialize a hundreds of empty fields back to the COBOL application with maybe one or two filled in just to get some data back. Needless to say this was a painful and error-prone process.

Once the web world had kind of taken over my team thought (foolishly) that we would start writing the logic on our side and deprecate the COBOL world. Unfortunately, development continued in the COBOL application. What we were able to do, though, was talk to the COBOL developers and get them to change the way we requested information. We found out that the "screens" we were sending back and forth did not, in fact, have to have identical fields. It was just how they had always done it when interfacing with the green screen.

After some discussion, we actually got a few of the new programs to utilize different fields for input and output. This even went so far as to breaking apart a few of the monolithic monsters that that gave us huge headaches and were pretty much impossible to test. (I'm talking QA testing, too. Unit/Integration tests were impossible.) This really helped speed up the development process and allowed a cleaner definition of what was required for input versus output. This was especially helpful for screens that returned collections. Before the change, we would have to send a request with (hundreds) of empty rows just so the COBOL program would have the correct data.

Oh, and this application is still being developed as of today. This isn't some horror story from the 90s.

CQRS versus the Command Pattern

Another thing to watch out for is mixing up ideas from the Command Pattern and CQRS. The commands in CQRS are not the same thing. Usually, in the command pattern, the command has an execute method on it that does the actual work. That is not how the commands work for CQRS. The commands should be built as Plain Old (Insert your development framework/language here) Objects. For us in the .NET world those are POCOs and, for Java devs, POJOs. (As was kindly pointed out in the comments: just a data structure.) They are just property bags that carry the intent of the user (a human or another system) to the domain.

Eventual Consistency

This is another thing that constantly gets tied to CQRS. Unfortunately, there is absolutely no reason that we have to have eventual consistency with CQRS. It is perfectly fine to inject a command handler into your web endpoint and pass in the command directly. We don't need a command bus to enjoy the magic of CQRS. And along with this, we also get:

Event Systems

And, for that matter, we don't have to have event systems in place to make use of CQRS. These are things that we can add to our system as we need to scale out and get a better feel for our domain. If these things are a requirement for your domain, obviously plan ahead and make extension points where you can plug these things in.

Why the confusion and over complication?

When I was able to make the shift over and try it out, plainly, without the overhead and complications I could see why. It is liberating. Once I broke everything apart and started practicing CQRS I could see how I could really empower my systems and make them even more decoupled by taking the next step. Adding in a command bus and event sourcing is a natural progression. Even so, we can implement a cheaper version of the two without a bunch of crazy overhead. This is made even more possible with (you guessed it!) F#. This will be the topic of my next post, so stay tuned!

Friday, January 24, 2014

JSON Web Token (JWT) with Simple.Web - Part 2 (Cleanup)

As I am trying to learn more and more about this F# thing and functional programming, in general, I realized that there was a way to cleanup my code from last post to make it more readable. By taking a hint from Scott Wlaschin's post on Railway Oriented Programming, I realized I could pull a lot of the pattern matching out of my code (where it makes sense) and move it into a bind function. Seeing it in my code reminded me a lot of factoring out common terms in math. In this case, we're factoring out the initial check of Some/None.

(As an aside, Scott's website F# for Fun and Profit is a fountain of knowledge and is worth spending a lot of time studying.)

So, we need a function that will do the Some/None checking for us and, since this is the common name everybody uses, we'll call it bind:

Edit: Thanks to Liam McLennan for pointing out that Option.bind can (should) be used instead of writing our own. (Great to have others reviewing my thoughts and it is much appreciated!)

This function takes a function and an Option<'a> as inputs. If the Option<'a> is Some, we take the value of it and bind it to the function f. If the Option<'a> is None, we just return None and our function f is never called.

I also like the infix operator that Scott uses and decided to use the same one for this version of bind. I'm keeping it all in the same module since it's specific to Option<'a> and fairly trivial to write out where it is needed. I'm sure in a larger project it would make sense to wrap all of these common functions into a helper module, but it is small enough.

Edit: And this is how the infix operator can be written using Option.bind:

So we can compose two functions together and into a third. Our new function takes whatever the input is for f1 and binds the output from f1 to f2, thus returning whatever f2 returns. If f1 returns None, then f2 is never called and None is returned from our new function. As such, that means f2 needs to return Option<'a> to keep the signature legal.

But why?

Why would we introduce this to our code? Like when we factor out terms when solving a math problem, we are (usually) reducing the complexity of the problem. If we take one of our original functions, we see that we have to test Option<'a>.isSome and Option<'a>.isNone. Not only do we have to test this in this function, we have to test it in all of them. When we factor this out, we reduce the complexity of our code and reduce the amount of repetitive tests. So, one of our old function:


In fact, the more functions we have where we're checking for Some/None, the more valuable our bind function becomes.

Cleaned-up code

So here is the same code as the last post but with the new bind function and operator.


I am always interested in hearing suggestions on how to make my code even better. Have something I could improve? Let me know!

Minor Update (26 Jan 2014)

I realized, after the fact, that I did not use the Simple.Web extension method to write the header. By forgetting this, the header would never actually get written because a NullReferenceException would be thrown. (The context.Response.Headers is initially null.) So, instead of instantiating the dictionary or anything crazy, we just use the extension method which takes care of everything for us. The good news is, this is actually much cleaner than the previous code.

Friday, January 17, 2014

JSON Web Token (JWT) with Simple.Web

I have been experimenting with Simple.Web and F# and trying to find different ways to exercise aspects of the framework. One of the things that is always annoying to me is writing authentication code. Fortunately, Simple.Web makes a distinction between how we authenticate users and how we track who is authenticated. I am not going to get into the Simple.Web handler that would actually authenticate the user, in this post, but am going to focus on how we can use JSON Web Tokens (JWT) in place of cookies and how we can leverage this within Simple.Web (using F# of course!).

Luckily for us, there are plenty of NuGet packages that implement JWT and so we can start there. I used the JWT package written by John Sheehan. It is open source and simple to use. I do have one nitpick about the package that I'll get to. (Read: I am too lazy at the moment to write my own solution since it's not really that hard.) So, we just need to "Install-Package JWT" to add it to our project.

JWT exposes some static methods wherein we pass our package we want serialized along with a "secret key" and we will receive a token we can pass back to the client. When the call comes back in, we pull the token out of the request headers and then can deserialize it. This is the annoyance with this package. We have two options to deserialize: JWT.JsonWebToken.Decode which returns a JSON string, or JWT.JsonWebToken.DecodeToObject which returns an object. My annoyance is that we get an object in the signature, but looking at the source code, the token we pass in is deserialized as a Dictionary. So, you have to cast to Dictionary to pull the values out. I would rather have this be a generic where I can specify what I want this to be deserialized into, or just make the signature Dictionary. As such, since I also use JSON.NET as my JSON serializer, I opted to have JWT return the JSON string and then delegated the object creation to JSON.NET. JSON.NET can also create our object without a public, default constructor which makes it nice to use with records. Enough of that, though.

Where's the code?

Moving onto the Simple.Web side of the house, we need to provide a Simple.Web.Authentication.IAuthenticationProvider which can be done in any IStartupTask. IAuthenticationProvider only has two methods on it, GetLoggedInUser and SetLoggedInUser. (This is how Simple.Web separates the idea of authenticating versus tracking the authentication.) GetLoggedInUser is given a Simple.Web.Http.IContext which contains references to the request and response objects. SetLoggedInUser also receives the context but also receives the IUser given from your authentication handler.

Since we need something to describe the types of things we're putting into our token, let's create a record to describe it.

So now we can track the user's id, name, and the expiration date of the token. We're using int64 here because it is easier to use DateTime.UtcNow.ToBinary() than it is to serialize the date. (At least this is the case with JWT which relies on JavaScriptSerializer.) I like the decomposition we can achieve with F#, so let's move now to write a little helper function that grabs the "Authorization" header from the request. We'll use Option to signal whether we find something or not.

Next, we need something that will pull the token out of the Authorization header, if it exists:

And then we need to turn that token into the Payload record we created above:

And, finally, let's take that Payload, if it exists, and turn it into an IUser if isn't expired.

Putting it all together

So now we can compose these functions and create our IAuthenticationProvider. One of my favorite things about F# is the ability to instantiate an interface without having to create a class. For such a simple interface, that works well here.

So this function takes in a "secret key" and returns a new IAuthenticationProvider. While I didn't cover it explicitly, the SetLoggedInUser should be simple enough to understand. All we need to do is tell Simple.Web to use it when a request needs to be authenticated. As I mentioned before, this can easily be done from an IStartupTask:

Authentication Module in Entirety

Edit: I have cleaned-up the code a little bit. The changes can be found here.


In under 60 lines of code we can put together a rudimentary JWT solution for our Simple.Web applications. When I get a moment, I will toss this up on GitHub so if you might have some improvements you can share them.

Saturday, January 11, 2014

Simple.Web F# Project Template Updates

F# Runtime Update

Thanks, again, to Dan Mohl for sharing some changes made to the ASP.NET MVC/WebAPI templates that I was able to apply to the Simple.Web F# project template. I had been digging into the template to figure out how to get allow changing the target F# runtime when you use the template to create a project but was unsuccessful. Fortunately, part of what Dan shared was a way to fix this-which I don't think I would have been able to figure out on my own. So, that's the first change made to the template.

Changes for Simple.Web Handlers

When I first started working with Simple.Web and tried using it with F# I ran into issues that made me have to build my handlers like so:

type Index() as this =
    member __.Get() = Status 200

    interface IGet with
        member __.Get() = this.Get()

I wasn't too happy about having to have so much duplication and double dispatch back to a handler's own methods, but most everything else worked out nicely that I just kind of accepted it, got used to it, and moved on. Once I had my first pull request into the main repository, Ryan Riley had commented on the pull request and asked why I was doing this. I'm glad he asked because I didn't even think about it. Luckily, I ran a little test and was able to pull out the extra method call!

type Index() = // No more aliasing required
    interface IGet with
        member __.Get() = Status 200

That cuts the lines of code, and complexity, in half, which is a huge win for F#. Again, it is nice to have another set of eyes out there actively looking at my work to help make it better.

Changes for Simple.Web NuGet Packages

One of the things I wanted to fix in this template was the fact that a C# file was automatically inserted into the project. This was due to the fact that I took a dependency on the Simple.Web.AspNet package. Fortunately, all this package does is reference Fix and Fix.AspNet. Since I have those packages available, I was able to safely remove the Simple.Web.AspNet package and just reference the other two directly. No more C# file to delete!

Update: Just to be a bit more clear about the Simple.Web.AspNet C# file, it was the OwinSetup file that usually goes with the C# project. The F# version has been with the F# Simple.Web template from the start.

Thanks so much to everybody who has downloaded, used, and provided feedback on this template.