Bad tests are worse than no tests. But never trying to test is far, far worse than writing bad tests.
Bad tests will exert a friction on your development efforts. They cost time to write, they cost time to maintain, they don't provide clear benefit, and they may even guide you to bad design choices. Under the desperate pressure of a rock-and-a-hard-place deadline situation, if you don't know how to write good tests it's often better to just not write them. You're naturally going to have more churn in the code, and removing the burden of testing may actually make the churn less disruptive and the code more stable.
Sidebar: "The desperate pressure of a rock-and-a-hard-place deadline situation" is a place you don't want to be. It very rapidly becomes an "unsustainable no-win situation." In the long term, the only way to win is not to play. In the short term, you do what you can to get by until it's over, and hope you'll be allowed clear the mess before moving on.
The key to Test Driven Design is that good tests drive good design. Bad tests don't... in the short term. Bad tests must be suffered, and examined, then removed, and rewritten. Bad tests must be improved through iteration, in tandem with the code that they test. Bad tests are learning tools to help you write good tests, which help you write better code. But if you don't have the time necessary to reflect, iterate, and learn from your tests, all you'll get from them is a creeping bitrot and a demoralizing stink.
The moral of this story is not that you shouldn't test. It is to recognize that tests are not inherently good. They are a tool with a particular purpose, and they must be used properly and given room to do their job. If you can't give them the room they need, then they're not the right tool for your situation.
Yeah, that's a situation that you wouldn't have in an ideal world. Maybe you should say something about it. Maybe you should find a new job. Or maybe it's just a tight spot and you need to fight through it. But for goodness sake, don't make a bad situation worse by imposing a burdensome constraint that you can't benefit from.
Among many other attributes, a professional is someone with enough pride of craft to insist on doing the best work that they can muster, and enough personal humility to accept that it will still often be wrong so they can learn from the failure.
Be suspicious of guidance that spends a large proportion of time addressing concerns of structure and organization, compared to what it spends discussing how to solve problems and make decisions.
Structure, organization, and naming are all shallow concerns that are easy to grasp onto and remember. They are easy to pass on. It is easy to attribute success to them through a very rational, but shallow analysis. And because they are easy, they get the focus and attention, while the kernel of wisdom that inspired the guidance is obscured, languishes, or is forgotten.
In my experience, guidance that focuses on shallow trappings like structure, organization, and naming tends to be the result of shallow thinking, and worse, tends to encourage shallow thinking. It espouses ideals that are theoretically beneficial. Often it will describe what it protects you from, but usually it offers scant evidence of how it helps you solve problems. And it might just make you feel like you're doing it wrong if you feel constrained by it.
Instead, prefer guidance that shows you how to think, rather than how to constrain your options. Prefer guidance that walks you through how to solve a problem, and shows you why thinking about a particular set of things will help lead you to a clean solution. And hold dear guidance that openly admits when it doesn't apply, and tells you why.
To say this all more elegantly:
Don't seek to follow in the footsteps of the wise. Instead, seek what they sought.
-- Matsuo Basho
A post model is not a view model. A form view model is a subset of the post model. Even when they are structurally the same, the roles are different, and it's highly likely that changes will exert a pressure to evolve the details of those roles separately.
Here's the problem: If, at the point that your view model and your post model happen to be structurally identical, you decide that it's "overkill" or "duplication" or "unnecessary complexity", and just use a single model, then you make those two roles completely implicit, and model a scenario that is only circumstantially true. A less experienced developer will come along, and absent the experience to know that there are two roles in one place, evolve the single object in-place rather than split the roles out. They don't know any better, and you have hidden from them the information that would indicate to them the better way forward.
In short a data object is incomplete without its context. Its responsibility, in terms of the Single Responsibility Principle, is naturally coupled from its context. This isn't a bad thing, as long as you recognize and respect that a data object has no standalone identity.
So how do you respect this symbiotic state of affairs? In a strongly-typed language it's very simple: by giving it its own, layer-specific type/class. This way it will become blatantly obvious when a data object has gotten too far from home and is in danger of being misused or misinterpreted.
In a loosely-typed language, you respect the coupling by cloning or mapping a data object as it moves between layers. Since there's no type system to stop you, it's extremely easy to just pass the same thing around until a clear requirement of change appears. But the very fact of the data object's implicit role means that the developer will have precious few clues to this. By always cloning or mapping into a new object the potential damage is mitigated, and the developer gets a clue that something is being protected from external modification. Hopefully this distinction of identity encourages the developer to make even small changes to structure and naming that ensure that the object is always expressive and clear of purpose relating to the context of its usage.
Recognizing this is a nuanced design skill, and it's one that people often resist in order to avoid adding more types to their programs. But it doesn't take long to reach a level of complexity where the benefits of this care and attention to expressiveness and signalling become clear.
POCO noun
1. A Plain Old CLR Object that does not derive from a base class, or implement an interface, which is defined as an integration point for a framework.
Victory
Infrastructure frameworks often brag these days about being POCO-compatible. This is in some ways a victory, because it used to be quite the opposite. In order to use an ORM, or an MVC framework, or really anything involving a "model", you needed to inherit some framework base class before it would recognize and respect the role of these objects in your application. But we're past that now. Today you can use POCOs even with frameworks that come directly from Microsoft, like the Entity Framework.
"Convention over configuration" is a similar principle of framework and infrastructure design that has been lauded for a few years now, and is finally hitting more mainstream shops. It trades on the ability to reduce the number of design decision points, allowing you to write less code, and do it faster. In other words, why take all the time and effort to deal with a complicated configuration mechanism capable of expressing myriad options, and then a complicated engine capable of processing all those options, when any one project is only going to use a slice of it all. With convention-based frameworks, just follow some simple rules, and let the framework handle all the details.
Tradeoff
For a long time, I was really happy about this. But today I realized that for all the benefits of simplification and boilerplate elimination, there are some pretty serious tradeoffs being made. Yes, the benefits can be absolutely huge. The gains in consistency, speed, reliability, and flexibility can't be overstated. It saves you from thinking and worrying about a lot of things that are fairly arbitrary in the context of the larger business problem.
Yet for all my appreciation, some shine is starting to come off this penny for one big reason I like to call "context-smearing". Both POCOs and conventions gain their structural homogeneity and rapid development by diluting object responsibilities, and trading away discoverability and expressive design. They take contextual info that ought to be localized, and smear it out between layers, across multiple or even many different locations in code. I am becoming increasingly careful about making this trade-off. I have seen the price it exacts, and I think there are definitely cases where that cost is probably not worth the gain.
Danger
So the big problem with both POCOs and conventions is a loss of context. What's so bad about that? In a word: coupling. Regardless of the absence of tangible framework attachments like base classes and interfaces, your POCOs and conventional objects are still tightly coupled to the framework. But instead of looking at the object and seeing the coupling, it is made almost completely invisible.
When you write a POCO that a framework needs to process, it needs to be simple. Anything too complicated and the framework won't understand it or will make an assumption that doesn't hold, and then the whole thing comes crashing down. Under this pressure your POCOs get dumber and dumber. Unless you're very diligent, before long they are not much more than simple property bags. And a bag of properties is no model. It's just a bunch of data.
With the behavior driven out, there's no context with which to understand the meaning of the data, the relationships, and the references. There's no indication why this property is reproduced over there, or why that container seems to exist solely to hold a single other object. Instead, this knowledge is encoded into all the places where the objects are used. It is distributed and diffused throughout the application, so that in no one place--or even handful of places--can you look to see the true meaning of what that object's role is.
The consequence of this is that points of consumption become fountains of boilerplate and redundancy in your code. At least it's consistent, though. So naturally you'll turn to conventions in order to scrub away some of this homogeneous noise. A good convention-based framework can look at your model, look at your usage, and determine all it needs to know to glue them together. This really streamlines things. But soon enough you've compounded the problem, because conventions actually encourage the exact same diffusion of context. Only conventions grind context down to an even finer sprinkling of dust, and then scatter it on the wind.
It should be clear enough why this is dangerous. It's very easy to slip from here into what John Cook calls "holographic code", which he describes far more elegantly than I have here.
Mitigation
In my opinion, none of this means you should shun POCOs and conventions. Heck, I'm even working on a convention-based Javascript framework of my own called copper.js. They do provide real benefits, especially when the ability to iterate rapidly and share coding and design responsibilities is more important than hi-fi business modeling. Both are highly effective tools in the context of such priorities. And lets be honest, that's the environment that most programmers are working in.
But take the sales pitch with a grain of salt. Conventions will make some of your code simpler, but they can also easily obscure the problem and the solution--or even just the actual model--behind the patterns mandated by the mechanism. An experienced "framework X developer" can see right past that noise. But should we really need to be framework experts just to understand the business problems and solutions? Solid and inviting framework documentation can go a long way in helping people see the signal in all the noise.
Likewise, POCOs are tremendously light-weight, quick, and flexibile. This commonly comes at a cost of design confusion or opacity of purpose when seen separated from their points of consumption. Establishing context is crucial in avoiding confusion and misuse. POCOs should be small and focused, not general and reusable. And they should be located as close as possible to the consumers.
Acceptance
While the problem could get very bad if you're not careful, it's rarely inevitable. And the speed you gain for the 80% case can often be invaluable. You just need to be aware of the pathologies in play, and take steps to mitigate them. If you're lucky, wise decisions on the part of the framework designers can reduce the number of compromises you have to make. But even more important--because it's actually under your control--wise decisions on the part of your development team can contextualize the anemic and opaque bits, limiting their reach.
My advice is to be aware of the problem, and tackle it head-on. Whether by careful coding or clear documentation, be diligent in providing context where the frameworks will tend to suck it away.
Ian Griffiths has written a very deep and informative post about what WinRT really is, and what is meant by all this "projection" vs "native" business. It's a great read, and you should dig in. Especially if you're an old COM hand, like myself.
I have seen a lot of important questions floating around lately (most notably from @kellabyte) that this post answers. But it takes a long time and many diversions before it gets there. So for those who don't have the time or the background to wade through Ian's post, I humbly present a TL;DR bullet list summary.
WinRT appears to be the new Windows API. It is probably the next evolution of Win32.
WinRT is native, and just as fast as native has always been.
WinRT is rooted in the next evolution of COM, not .NET
WinRT involves just as much indirection (and "inefficiency", compared to COM-less native code) as COM always has.
WinRT "projections" are language-specific. For C++, they're native.
WinRT projections do no more and no less than smooth over much of the API for consuming languages, putting a curtain around the pain and noise of working with COM and HRESULTs directly.
WinRT projections are not the next .NET windowing framework. They do not replace WinForms, WPF, Silverlight.
I'll also add to this list my own prediction: We will probably not want to directly consume WinRT any more than we wanted to directly consume Win32. There will be value in a GUI framework to abstract that away, just as there always has been. I haven't heard any mention of the Metro-based successor to WPF/Silverlight, but I would expect one to be delivered with the next version of VS and .NET. I look forward to hearing more from DevDiv now that all the WinRT details are starting to trickle out.
I've been playing a little bit lately with AngularJS. Angular is one of a spate of new JavaScript frameworks that provide powerful tools for building interactive web sites and web apps by capitalizing on mature patterns and practices that have served well in the "native client" space. Most of these frameworks provide some sort of separation of concerns between the "view" and the "model".
The "big four" frameworks in this space are AngularJS, KnockoutJS, Backbone.js, and Batman.js. All four of these frameworks are primarily concerned with finding ways to purify and simplify two parts of the architecture of most modern applications: the UI (a.k.a. view, in this case, the HTML), and the model that sources the information in the UI. The primary obstacle in the way of achieving this goal is something called data-binding, which is a terse and useful term for synchronization of the dynamic information displayed in your UI to the state of an object model in your application.
AngularJS goes about this in a strikingly different way from the other three frameworks. The developers of Angular elected to use HTML "templating" as the mechanism of data-binding. Which is to say, you take a "template" of your web page, poke holes in it where you want your dynamic information to go, and then place in those holes small bits of syntax which tell the templating engine how to build the HTML to fill the holes at runtime. Below is a simple example of a template, as it would appear in an application built with AngularJS. If it looks familiar, that's because it's my own modified version of the front page demo from the AngularJS website. Be sure to note the fidelity of the template to the HTML it will produce. It's very clean, and you can see clearly the features of the document in it.
Templating is great for producing a document on demand using data that isn't known until runtime. It's a technique that has been around for quite a while now, and we've mostly figured out some really convenient syntax and conventions for doing it. Heck, ASP.NET MVC is a template-based solution for creating HTML on the fly. It works fantastic. But MVC is not dynamic once that document leaves the server. It builds the HTML once and sends it off to the client, never to revisit it again. The next document it serves up will be a whole new document in response to a whole new request. This is very different from the way Angular uses templating on the client, where the same document may be changed and re-rendered hundreds of times, or more.
Here I feel this solution starts to strain, and this bothers me about Angular as a general solution for web apps. Let me be clear, I think it may be a fantastic way of building static pages, interactive forms with a dash of business logic, or generally anything where there isn't need for complex logic, or generation of large swaths of HTML from nothing. But as pages become more dynamic after the initial rendering, the templates can become bloated and less emblematic of the produced document structure. Today web pages are commonly less about content, layout, and flow, and increasingly about interactivity, dynamism, and tasks. This impacts a templating solution in that expressions expand and dominate, and eventually replace, their HTML content containers. The bulk of logic starts to move further away from the template and deep into javascript and framework internals start to leak out. As an example, see the introduction page on creating your own "custom elements" in AngularJS. We've obviously stepped far away from a "templating" solution here.
For extremely dynamic sites where most of the elements on the page are subject to change in some way or another and very little can be considered "static", I think more conventional solutions are more appropriate. At that point, it becomes appropriate to move logic as much as possible into code, where it can be encapsulated in objects and layers with clear roles and identities. Here it may be prudent to start considering Backbone, Batman, Knockout, or maybe even something else.
For myself, I'd love to see a JavaScript framework that employed MVVM in an unobstrusive way, by treating the HTML document as a passive view which is bound to at run time. I can always drop in Mustache templates for creating HTML in small bits where needed. And "declarative bindings" can just as easily be achieved outside of the HTML, via a nice fluent API. Does anyone know of a framework like that?
First I want to say thanks to Alex for the feedback on my previous post. I did eventually manage to clean up my design quite a bit, but couldn't really put my finger on why it was so hard. After reading Alex's comment it occured to me that I hadn't really thought very hard on where the layer boundaries are. I had (and maybe still have) my persistence, my service, and my infrastructure layers somewhat intertwined. I'm ashamed to have been bitten by this considering how much of a proponent of intentional design and layering as I've been. But I was on a spike to tackle some unfamiliar territory, as well as fighting with a bit of an uncooperative helper framework, so it seems I bit off more than I could chew.
The problem I was trying to deal with was a multi-threaded service, with some threads producing data, and others consuming it to store to a DB. Between device sessions, threads, DB sessions, and domain transactions, I had a lot of "resources" and "lifetimes" that needed very careful managing. It seemed like the perfect opportunity to test the limits of the pattern I've been fond of lately, of creating objects whose primary responsibility is managing lifetime. All services which operate within the context of such a lifetime must then take an instance of the lifetime object. This is all well and good, until you start confusing the context scoping rules.
Below is a sample of my code. I'd have to post the whole project to really give a picture of the concentric scopes, and that would be at least overwhelming and at worst a violation of my company's IP policies. So this contains just the skeleton of the persistence bits.
The important part of this picture is the following dependency chain (from dependent to dependency): Repository->DataContext->SessionProvider->UnitOfWork. What this would indicate is that before you can get a Repository object, you must have a Unit Of Work first. But it's common in our web apps to basically just make the UoW a singleton, created at the beginning of a request, and disposed at the end. Whereas in a service or desktop app the most obvious/direct implementation path is to have different transactions occurring all over the place, with no such implicit static state to indicate where one ends and the next begins. You get a common workflow that looks like this: Create UoW, save data to Repo, Commit UoW. This doesn't work nicely with a naive IoC setup because the UoW, being a per-dependency object now, is created after the Repo, which already has its own UoW. The changes are in the latter, but it's the former that gets committed. So either the UoW must be created at a higher scope, or the Repo must be created at a lower scope, such as via a separate factory taking a UoW as an argument. I'm not super fond of this because it causes the UoW to sort of vanish from the code where it is most relevant.
One final option is that the dependency chain must change. In my case, I realized that what I really had was an implicit Domain Transaction or Domain Command that wasn't following the pattern of having an object to manage lifetime scope. The fact that this transaction scope was entirely implicit, and its state split across multiple levels of the existing scope hierarchy, was one of the things that was causing me so much confusion. So I solved the issue by setting up my IoC to allow one UoW and one Repo per "lifetime scope" (Autofac's term for child container). Then I created a new domain command object for the operation which would take those two as dependencies. Finally the key: I configured Autofac to spawn a new "lifetime scope" each time one of these domain command objects is created.
Again this illustration is a bit simplified because I was also working with context objects for threads and device sessions as well, all nested in a very particular way to ensure that data flows properly at precise timings. Building disposable scopes upon disposable scopes upon disposable scopes is what got me confused.
I have to say that at this point, I feel like the composability of the model is good, now that I've got layer responsibilities separated. But that it feels like a lot is still too implicit. There is behavior that has moved from being explicit, if complex, code within one object, to being emergent from the composition of multiple objects. I'm not convinced yet that this is an inherently bad thing, but one thing is for certain: it's not going to be covered properly with just unit tests. If this is a pattern I want to continue to follow, I need to get serious about behavioral/integration testing.
I'm noticing a strange thing happening as I make classes more composable and immutable. Especially as I allow object lifetime to carry more meaning, in the form of context or transaction objects. This shift has made object initialization involve a lot of "real" computation, querying, etc. Combined with a desire to avoid explicit factories where possible, to capitalize on the power of my IoC container, this has led to an accumulation of logic in and around constructors.
I'm not sure how I feel about so much logic in constructors. I remember being told, I can't remember when or where, that constructors should not contain complex logic. And while I wouldn't call this complex, it certainly is crucial, core logic, such as querying the database or spinning up threads. It feels like maybe the wrong place for the work to happen, but I can't put my finger on why.
It's important to me to be deliberate, and I do have a definite reason for heading down this path. So I don't want to turn away from it without a definite reason. Plus, pulling the logic out into a factory almost certainly means writing a lot more code, as it steps in between my constructors and the IoC container, requiring manual pass-through of dependencies.
I'm not certain what the alternative is. I like the idea of object lifetimes being meaningful and bestowing context to the things below them in the object graph. Especially if it can be done via child containers and lifetime scoping as supported by the IoC container. But it is really causing me some headaches right now.
My guess at this point is that I am doing too much "asking" in my constructors, and not enough telling. I've done this because to do the telling in a factory would mean I need to explicitly provide some of the constructor parameters. But I don't want to lose the IoC's help for providing the remainder. So I think I need to start figuring out what my IoC can really do as far as currying the constructor parameters not involved in the logic, so that I can have the best of both worlds.
Maybe it will bite me in the end and I'll pull back to a more traditional, less composable strategy. Maybe the headaches are a necessary consequence of my strategy. But I think it's worth trying. At least if I fail I will know exactly why not to do this, and I can move on to wholeheartedly looking for alternatives.
I've been working with Jeremy D Miller's StructureMap IoC container lately. Over the weekend I also did some research on Castle Windsor, in order to properly offer an opinion to Krysztof Kozmic about whether he should remove nested containers from it. Previous to all this, the only IoC I'd really had the opportunity to use on a real project was Autofac. Maybe it's just the bias of first exposure, but I still like Autofac the best. But I can point to one big tangible reason that I like Autofac better, which the other two options don't have: lifetime scopes.
Lifetime scopes are the primary unit of context in Autofac. When the framework has to resolve a dependency, it doesn't pull it from a container. Rather, it pulls it from a lifetime scope. A lifetime scope is different from a container in that you can't change or query its configuration at runtime. It is a purely resolution-oriented interface. But more importantly, in concert with a few other features, it offers a very convenient way to establish what I call "resolution contexts" within your application.
A resolution context is a "bubble" within your application with some or all of its own resolution rules. You can dictate that within a given bubble, certain objects are guaranteed unique, and will be cleaned up when the context is disposed of. Or you can ensure that dependencies are resolvable inside the bubble, but not outside of it. By nesting bubbles that do this, you can enforce inter-layer dependency rules ensuring, for example, that the domain services layer does not depend on the UI layer. You can see the obvious reason that the word "scope" is used in the name: scoping is their explicit purpose.
The features that combine with lifetime scope to make it a truly powerful idiom are tags, and tagged registration. You can tag a lifetime scope with an object that identifies that scope. This value can also be specified in a given type registration to mark it as being unique within a scope having that tag.
Combined, these features allow you to tie instances of objects to a scope, and ensure that anything resolved within that scope gets a particular object. This can be particularly useful for managing things such as database transactions, or domain units of work. Or really any transaction-like operation consisting of a set of sub-operations involving resources which need only, or should only, exist for the duration of the parent operation.
Without this feature, it falls to the developer to explicitly write an object whose responsibility is to manage the lifetime or reach of these things, and another whose responsibility it is to generate and dispose of these scoping objects. This is essentially simple, but for an inexperienced dev it is a tough problem. For an experienced dev, it's annoyingly repetitive.
The thing I love about Autofac is that it supports tagged scopes which make it trivially easy to solve this problem. The thing I hate about Autofac is that you can't define as part of the registration process where you want these nested lifetime scopes to be generated. Instead, you must use a hook, a direct framework reference, a relationship type or some other extension mechanism, to dictate when nested lifetime scopes are introduced.
The reason I have these conflicting feelings is that Autofac approaches, but falls short of what I have come to consider the ideal role of an IoC container: to be the composer of the application. The use to which most of us put IoC containers today diverges from the implications of the name "container". While they do serve as containers in the sense that they hang onto shared (singleton) instances of objects, they also fill a factory role. And with Autofac's nested lifetime scopes, they can draw boundaries around self-contained operations and object graphs. Windsor prides itself on powerful, flexible "release" mechanisms, for cleaning up objects that are no longer needed. Again, this goes far beyond the role of a "container".
Yet despite this, the configuration API of most IoC frameworks still is highly container-oriented. We "register" types and interfaces, attaching bits of metadata here and there as needed. These are "stored" in the container, which can be queried or overridden at run-time by adding or removing registrations from the container.
But an IoC is at its most effective when it is acting not as a container, but as a rules engine, to regulate when new objects are created, and when existing ones are reused. Or how one object is chosen from many that implement a common interface. Or when and how object graphs are disposed of and resources cleaned up. These roles all build toward an overarching theme of "composing" the application. That is to say, building it up out of different blocks of functionality, fitting them together according to a pattern which is clearest when viewed as a whole.
This is why the ServiceLocator block from Microsoft makes me angry. While you can still configure the composition of your application using your IoC of choice, ServiceLocator exposes it as if it were a simple abstract-to-concrete mapping operation. And beyond that, the base idea of a centralized "service locator" completely undermines the goal of IoC in the first place, which is to invert the control. Don't ask for what you think you want. Just expose the dependency declaratively, and let it be provided to you as deemed appropriate by someone whose whole job is to know what you need.
I say we should embrace the composer idiom. I would love to see an IoC framework that took this role to heart, and wore it on its sleeve, so to speak, by expressing it naturally in its API. Autofac comes closest of any container I've used or researched, but it still shows some artifacts of its container heritage. One big improvement that could be made is a declarative configuration-time way of establishing when new scopes should be spun up. I posted a gist to this effect over the weekend and shared it with Krysztof Kozmic, hoping he'd take some inspiration from it for Windsor. But in truth, Windsor also still has a lot of other aspects that are tied to its identity as a "container".
Maybe it's time for someone to take another crack at the IoC problem. The ideal functionality is mostly all available out there. All that's lacking is an API that makes its true role obvious and natural.
It turns out I really can't bang this drum enough.
This weekend I tweeted three simple rules for determining if a class's responsibility has been properly assigned. Class responsibilities were on my mind for a significant portion of the day on Saturday. It wasn't anything conscious or intentional. Rather I think it was a sort of run-off from a lot of responsibility talk over the course of the previous week.
Lately I find that whenever someone asks my opinion about a particular design decision, the question of responsibility is the first place I go. It's far and away the most effective and powerful thing to ask, as it easily gets the most guaranteed bang for the buck.
At both the class and the method level, the most important factor in deciding where and how to implement a particular bit of functionality is whose responsibility it is. Likely, the whole picture of the task is made up of several bits of responsibility that belong in several places. If your application is architected in such a way that responsibilities are well isolated and assigned to layers and clusters of classes, then breaking the task down into those constituent responsibilities gives you a draft blueprint for how to get the job done.
If your application is not fortunate enough to have such a clear and well articulated architectural scheme... well I'd advise you to make it a goal. Start by learning about onion architecture. But in the meantime, try to keep in mind the rules of thumb below and evaluate your classes against them as you work. If they don't measure up, see if they can be improved incrementally as you work. Leave them cleaner than you left them.
Now, without further ado, the rules.
If you can't sum up a class's responsibility in 2 sentences, shift your level of granularity. It almost certainly has more than one responsibility, and you need to think hard about why you've combined them.
If you can't sum up a class's responsibility in a single sentence, then you have lots of room for improvement. Don't be satisfied until you have a clear, concise statement of the class's responsibility.
If you can sum up a class's responsibility in a single sentence, go a step further and think hard about whether it has another unstated or assumed responsibility. Unit tests can really help here.
The message here is not subtle. Every class needs to have a clear answer to the question "what is my responsibility". This is, in my experience, the most important factor in establishing clean, clear, composable, testable, maintainable, comprehensible application design and architecture. It's not the end of the road, but it's a darn important step on the way.
I've been thinking a lot about the DRY principle lately. You know the one. DRY: Don't Repeat Yourself. It's a principle that was first made popular in the book "The Pragmatic Programmer". Unlike much of the rest of the content of the book, DRY is something that appears to have penetrated fairly deeply into the collective consciousness of the greater programming community. In a certain respect, this is a huge success on the part of Andy Hunt and David Thomas. To have disseminated so completely such a fundamental rule of programming is surely something to be proud of.
Note what I just said. DRY is "a fundamental rule of programming." I suspect very few people with appreciable experience would reasonably and honestly dispute this claim. But a rule is a funny thing, and a fundamental one doubly so. Rules are a first-pass at quality. They establish a minimum level of ensured quality by being a reasonable course of action in the vast majority of cases, at the expense of being a sub-optimal course of action in many cases, and even a plain bad option in some.
I am by no means saying that DRY is a bad thing. I'm not even saying not to do it. But I am saying that applying DRY is a beginner's first step toward good design. It's only the beginning of the road, not the end.
Eliminating repetition and redundancy is a very natural and obvious use for programming. It's especially hard to deny for so many people who slide into the profession by way of automating tedious and repetitious manual labor. It just makes sense. So when you tell a new programmer Don't Repeat Yourself, they embrace it whole-heartedly. Here is something that a) they believe in, b) they are good at, and c) will automatically achieve a better design. You'd better believe they are going to pick up that banner and wave it high and proud.
This is a good thing. If you can get your team to buy in to DRY, you know you'll never have to worry about updating the button ordering on that dialog box only to find that it hasn't changed in 3 scenarios because they're using a near identical copy in another namespace. You know that you'll never have to deal with the fallout of updating the logic that wraps up entered data for storage to the database and finding inconsistent records a week later because an alternate form had repeated the exact same logic.
What you might run into, however, is:
A method for rendering currencies as user-friendly text which is also used to format values for XML serialization. The goal being to keep all text serialization centralized regardless whether it's meant for display or storage.
Views, controllers, configuration, and services all using the same data objects, regardless of the dramatically different projections necessary for every operation. This in the interest of avoiding redundant structures.
You might find that your views, controllers, and persistence layers all depend directly on a single tax calculation class. Of course here the goal is very simply to centralize the business logic, but actively works against establishing a proper layering in the application.
These are all very sub-optimal, or even bad, design decisions. They are all examples that I have seen with my own eyes of decisions made in the name of DRY. But DRY itself is not the cause. The problem is that the people responsible for these decisions have unknowingly pitted DRY against other quality design rules and principles.
DRY focuses on behavior. That is to say algorithms. "Algorithms", along with "data", are the fundamental building blocks of applications. They are the substance our work is made of. But trying to build complex software while thinking only or primarily at this elementary level is like trying to map a forest one tree at a time.
At some point, the programmer must graduate from the rule of DRY to the nuance of responsibility. The Single Responsibility Principle (SRP) is the cardinal member of the SOLID family. It's premier position is not simply a coincidence of the acronym. It's also the next most important rule to layer onto your understanding of good design.
Responsibility is about more than function. It's also about context. It's about form and purpose. Identifying and isolating responsibilities allows you to take your common DRY'ed functionality, and cast it through various filters and projections to make it naturally and frictionlessly useful in the different places where it is needed. In a very strict sense, you've repeated some functionality, but you've also specialized it and eliminated an impedance mismatch.
Properly establishing and delimiting responsibilities will allow you to:
Take that currency rendering logic and recognize that readability and storage needs present dramatically different contexts with different needs.
See that the problem solved by a data structure is rarely as simple as just bundling data together. It also includes exposing that data in a form convenient to usage, which may vary dramatically from site to site.
Recognize that calculation is worth keeping DRY, but so is the responsibility to trigger such calculation. It can be both triggered and performed in the business layer, and the result projected along with contextual related business data to wherever it needs to be.
By layering responsibility resolution on top of the utilitarianism of DRY, you take a step back from the trees and can begin to manage a slightly wider view of the problem and solution spaces. This is the key and crucial lesson that all beginning programmers must learn, once they've mastered the art of DRY. Once again, DRY is fundamental and indispensable. It's one step along the path to wisdom and, in general, should not be skipped or omitted. But it's not an end in itself. So don't stop when you get DRY. Start looking for the next step. And consider responsibility resolution as a strong candidate.
You are faced with an unfamiliar coding problem. You know what you're "supposed to do"--use this pattern, follow that convention--but you don't know why. And the guidance seems ever so slightly... off. The context is just a bit different. The constraints are a bit tighter here. A bit looser there. The peg doesn't quite fit in the hole, but everyone is telling you it should.
Everyone is telling you "this is how you do it" or "this is how we've always done it". You think you can follow the rule, but you don't think it will be pretty. What do you do?
I have witnessed many people in this situation make their choice. Most commonly they do whatever it takes to follow the rule. If the peg doesn't fit, they pound it till it's the right shape. If the cloth won't cover, they'll stretch, fold, and tear creatively until they can make a seam. Somewhat less often, they just toss out the advice and do it the way they are comfortable with. Often that just means "hacking out" an ad hoc solution.
In either case, the person learns nothing. They will repeat their struggle the next time they face a situation that is ever so slightly different. They will sweat, stress, and hack again. Over and over, until someone shows them a new rule to follow.
As students we are taught to learn first by rote, then by rule. If we are lucky, we are eventually tested on whether we have learned to think. And most commonly we manage to slog it out and do well enough to pass without actually being required to think. It is so very easy to become comfortable with this model of learning. A particular someone, vested with the responsibility of fertilizing our minds and nurturing the growth of understanding, knows the answers. We don't understand, but someone can at least tell us when we are right or wrong, and often that's enough.
We settle into dependence. And by settling, we establish roadblocks in our path before we even set foot on the road. By settling, we refuse to take ownership of our own knowledge. We put ourselves at the mercy of those who know more and are willing to share of their time and understanding to help us overcome obstacles of our own creation.
This situation is not inevitable. You can avoid dooming yourself to toil under it with a very simple determination. But make no mistake, this simple determination will require determination. When you face the prospect of doing something that you do not understand, stop, take note, and ask "Why?" Refuse to continue on with any course of action until you know why you are doing it.
I'm not talking about questioning authority here (though that's good too). I am advocating understanding. If you think there's any chance you may have to face a similar situation again, then as a professional developer it behooves you to understand what you're doing this time, and why. This prepares you firstly to defend your actions, and secondly to tackle similar but different problems in the future.
By recognizing the reasoning behind a particular prescribed course of action, when you encounter a similar situation in the future you will be able to identify the subset of the problem that is subject to the prescription. Seeing this allows you to conceptually distinguish that part of the problem from the rest. From this vantage point you can decide whether the remainder is just noise that has to be accommodated, or something more significant. You will be able to start to consider whether there is another layer or dimension to the problem which might be better served by a different or additional pattern. You will be able to think, intelligently, and intentionally, about the problem both as a whole, and in part.
Lack of mindfulness is the scourge of intellectual pursuits (and a great many other things in life). Whether in programming, in health, in investment, etc., it binds you to the service of rules and systems. It puts you at the mercy of those who have understanding, and under the thumb of those who own the systems. Benevolent or otherwise, do you really want your own success and satisfaction to come at the whim of someone else, for no other reason than that you couldn't be bothered to put in the effort to understand? Do you want to spend your career tromping the same grounds over and over again, never coming to any familiarity or understanding of the landscape?
Always ask, "Why?" Then take the time to understand. Always be deliberate and intentional about applying any solution. Don't just follow the directions of the crowd, or some authority. You're not a patient taking orders from your doctor. This is your domain. Own your knowledge. Your future self will thank you.
The Single Responsibility Principle (SRP) is a crucial tool in your toolbox for managing complexity. Bob Martin has a great essay on the Single Responsibility principle which expresses one of the biggest benefits that it can deliver to you. The SRP is predicated upon the unfortunate reality that changing code is the biggest source of bugs. By extension, the easiest way to avoid bugs is to ensure that whenever you have to make a change, it affects as little of the code as possible.
This is well known among experienced developers, but as Martin notes at the end of his essay, it's extremely difficult to get right. In my experience, very few devs take the principle as far as they should. Especially considering the fact that most of us were taught that Object-Oriented Design is all about bundling up code that works together, it can be easy to get lulled into confidence about what it takes to truly adhere to SRP.
Martin says, "conjoining responsibilities" comes naturally to us. Programmers are finely-honed pattern matching machines. It often comes far too easily. Most of us begin our careers weaving responsibilities and dependencies throughout our code, in the interest of solving a problem as quickly and efficiently as possible... with bonus points for being clever. Then we start learning the SOLID principles and start to adjust our coding habits. We take SRP to heart and break up responsibilities along reasonable lines, even if it means writing a bit more code, and synchronizing state that could theoretically be shared if all things were equal.
Then we stop.
We move our logging out of exception handlers and into service classes. But we leave severity conditions and entry formatting mixed in with the filestream management. We separate our business logic from our UI rendering and input handling. But we pile up unrelated domain operations in our controllers and presentation models. We shrink our class sizes from 1500 lines to 500 and claim victory.
This is not enough.
For one thing, responsibilities that often seem naturally elemental can usually be broken down yet further. The log example is a perfect one. File stream management is a responsibility complex enough to be given its own class. Text formatting is yet another. And severity handling is something that can be configured apart from the other two aspects. Oh, and don't forget that interfacing with an unmockable framework resource class such as System.IO.Filestream is a worthy role all its own. Each of these is a responsibility that can be wrapped up in a small class of just 100 or so lines, and exposed with a simple fine-grained interface of just a few methods. Compose these together and you have a logging service that's flexible, highly testable in all aspects, and most importantly, can evolve and be maintained independently along several orthogonal dimensions, without interfering with the other functionality. And on top of all this, you get the automatic benefit that it's much more friendly to dependency injection and unit testing.
The other important lesson to learn is that SRP doesn't just apply to classes but also to methods. It's almost never necessary for a method to be more than 30 or so lines long, on the outside. A method of such restricted size will inevitably have fewer arguments to worry about, for starters. And further, it almost inherently prevents spaghetti code. Purely by the act of breaking out small operations of a handful of steps apiece, and giving each a highly specific and expressive name, you can avoid the "flying V" of nested loops and conditionals. You can avoid long-lived context variables and status flags. You can avoid stretching tightly coupled cause and effect relationships across multiple screens-worth of code. And you'll likely find yet more places to naturally slice up a class into separate responsibilities.
All of these help to control sprawling complexity. This sounds unintuitive, because if you follow the rough rule of 30 lines per method, 200 lines per class, you'll almost certainly end up writing far more classes, and probably more code in general. But you will always know exactly where to find code related to any particular thing that may go wrong. And you can always be certain that the portion of the application that is affected by a change to a particular unit of functionality will be highly constrained by virtue of the reduced number of interactions and dependencies that any one unit needs to worry about.
Consider what you can do to take the next step with SRP. Don't be satisfied with the first-order effects of a naive application of the principle. Rededicate yourself, try it out, shrink your units, and see the benefits in your code.
Many words have been written in the TDD community about the myriad ways of mocking in service of unit tests. After all this effort, there remains a great deal of confusion, ambiguity, in the understanding of many--maybe even most--developers who are using mocks.
No less than the likes of the eminently wise Martin Fowler has tackled the subject. Fowler's article is indispensible, and it in large part built the foundation of my own understanding of the topic. But it is quite long, and was originally written several years ago, when mocks were almost exclusively hand-rolled, or created with the record/replay idiom that was popular in mocking frameworks before lambdas and expressions were added to C# and VB.NET with Visual Studio 2008. Add to that the fact that the article was written in the context of a long-standing argument between two different philosophies of mocking.
Unfortunately these arguments continue on even today, as can be seen in the strongly-worded post that Karl Seguin wrote last week. Looking back now, with several more years of community experience and wisdom in unit testing and mocking behind us, we can bring a bit more perspective to the discussion than what was available at that time. But we won't throw away Fowler's post completely. Within his post, there are firm foundations we can build on, in the definitions of the different types of mocks that Fowler identified.
There are four primary types of test doubles. We'll start with the simplest, and move through in order of ascending complexity.
Dummies
A dummy is probably the most common type of test double. It is a "dumb" object that has no real behavior. Methods and setters may be called without any exception, but without any side-effect, and getters will return default values. Dummies are typically used as placeholders to fill an argument or property of a specific type that won't actually be used by the test subject during the test in question. While a "real" object wouldn't actually be used, an instance of a concrete type may have strings attached, such as dependencies of its own, that would make the test setup difficult or noisy.
Dummmies are most efficiently created using a mock framework. These frameworks will typically allow a mock to be created without actually configuring any of the members. Instead they will provide sensible defaults, should some innocuous behavior be necessary to satisfy the subject.
Stubs
A stub is a test double which serves up "indirect input" to the test subject. An indirect input is information that is not provided to an object by the caller of its methods or properties, but rather in response to a method call or property access by the subject itself, to one of its dependencies. An example of this would be the result of a factory creation method. Factories are a type of dependency that is quite commonly replaced by a stub. Their whole purpose is to serve up indirect input, toward the goal of avoiding having to provide the product directly when it may not be available at the time.
Stubs tend to be quite easy to set up even with more primitive mocking frameworks. Typically, all that is needed is to specify ahead of time the value that should be returned in response to a particular call. The usual simplicity of stubs should not be taken as false comfort that the doubles are not too complicated, however. Stubs can get quite complex if they need to yield a variety of different objects multiple calls. The setup for this kind of scenario can get messy quick, and that should be taken as a sign to move on to a more complex type of double.
Mocks
A mock is a type of test double that is designed to accept and verify "indirect output" from the subject class. An indirect output is a piece of information that is provided by the test subject to one of its dependencies, rather than as a return value to the caller. For example, a class that calls Console.WriteLine with a message for printing to the screen is providing an indirect output to that method.
The term "mock" for a particular type of test double is in a certain way unfortunate. In the beginning there was no differentiation. All doubles were mocks. And all the frameworks that facilitated easy double creation were called mocking frameworks. The reason that "mock" has stuck as a particular type of double is because in those beginning times, most test doubles tended to take a form close to what we today still call a "mock". Mocks were used primarily to specify an expectation of a particular series of method calls and property access.
These "behavioral mocks", or "classical mocks" as Fowler calls them, gave birth to the record/replay idiom for mock configuration that reached its peak in the days of RhinoMocks. And due to the tendency of inexperienced developers to create complicated object interactions and temporal coupling, mocks continue to be a very popular and common form of test double. Mocking frameworks make it far easier to unit test classes that rely on these types of coupling. This has led many to call for the abolishment of mocks and mocking frameworks in a general sense, claiming that they provide a crutch that makes it too easy to leave bad code in place. I'm sympathetic to the sentiment, but I think that this is throwing the baby out with the bathwater.
Fakes
Fakes are the most complicated style of test double. A fake is an object that acts simultaneously as both a stub and a mock, providing bidirectional interaction with the test subject. Often fakes are used to provide a substantial portion of the dependency's interface, or even all of it. This can be quite useful in the case of a database dependency, for example, or a disk storage service. Properly testing an object that makes use of storage or persistence mechanisms often requires testing a full cycle of behavior which includes both pushing to and pulling from the storage. An in-memory fake implementation is often a very effective way of avoiding relying on such stateful storage in your tests.
Given their usefulness, fakes are also probably the most misused type of test double. I say this because many people create fakes using a mocking framework, thinking they are creating simple mocks. Or worse, they knowingly implement a full-fledged fake using closures around the test's local variables. Unfortunately, due to the verbosity of mocking APIs in static languages, this can very easily become longer and more complex code than an explicit test-specific implementation of the interface/base class would be. Working with very noisy, complicated, and fragile test setup is dangerous, because it's too easy to lose track of what is going on and end up with false-passes. When your test's "arrange" step starts to overshadow the "act" and the "assert" steps, it's time to consider writing a "hand-rolled fake". Hand-rolled fakes not only remove brittle and probably redundant setup from your tests, but they also often can be very effectively reused throughout all the tests for a given class, or even multiple classes.
It's not Just Academic
These are the primary categories into which nearly all, if not all, test doubles can be grouped. Fowler did a great job of identifying the categories, but I think this crucial information is buried within a lot of context-setting and illustration that doesn't necessarily offer great value today. Mocking is ubiquitous among the subset of developers that are doing unit testing. But too many people go about unit testing in an ad hoc fashion, rather than deliberately with a plan and a system for making sense of things. I believe that a simple explanation of the major types and usages of test doubles, as I've tried to provide here, can aid greatly in bringing consistency and clarity of intent to developers' unit tests. At the very least, I hope it can instill some confidence that, with a little discipline, pattern and reason can be found in the often messy and overwhelming world of unit testing.
Have you ever proposed a code change or a course of action in a project for the purpose of improving the stability and maintainability of the code base, only to have someone dispute the need on the basis of
? I was flummoxed the first time this happened to me. Since then I've learned that it's not at all rare, and in fact may even be common.
The YAGNI principle is a wonderful thing. Used properly, it can have a huge beneficial impact on your productivity, your schedule, and on the maintainability of your product. But like so many other important ideas in the history of software development, YAGNI has become a poorly understood and misused victim of its fame. Through constant abuse it has become difficult to communicate the sentiment that it was intended for without a thorough explanation. And I can't count the number of times I've heard YAGNI cited in a completely incorrect or even dangerous way.
The term "YAGNI" has fallen prey to a similar disease as "agile". People often invoke it as an excuse not to do something that they don't want to do. Unfortunately, this quite often includes things that they
should
do. Things that have long constituted good design, and good software engineering practice. A few examples of things that I have personally been horrified to hear disputed on YAGNI grounds include:
These are all activities that are strongly valued and diligently practiced in the most productive, successful, and small-A-agile software development organizations and communities. For myself and many of you out there, it's patently obvious that this is a subversion and abuse of the YAGNI strategy. Your first instinct in response to this kind of misuse is to say, with no little conviction, "that's
not
what YAGNI means."
This, of course, will not convince anyone who has actually attempted to use the YAGNI defense to avoid good engineering practices. But to refute them, you needn't rely solely on the forceful recitation of the principle as they do. Fortunately for us, YAGNI is not an elemental, indivisible tenet of software engineering. It did not spring fully-formed from Ron Jeffries' head. Rather it is based in experience, observation, and analysis.
on the XP page that the key to sensible use of the YAGNI principle is remembering that it tells you not to add something that you think you might or probably will need, or even certainly will need, in the future. YAGNI is a response to the urge to add complexity that is not bringing you closer to the immediate goal. Particularly common instances of true YAGNI center on features that haven't been identified as either crucial or wanted, such as configurability, alternative logging mechanisms, or remote notifications.
Looking at my original list, it is clear that none of these things truly add complexity in this way. A very naive metric of complexity such as "number of code entities" may seem to indicate the opposite. But these are actually all established and reliable methods for
controlling
complexity. What these techniques all have in common is that they restrict the ways in which parts of your program are allowed to interact with other parts of your program. The interaction graph is the most dangerous place for complexity to manifest in a program because it compounds the difficulty of changing any one part of the application without affecting the rest of it like a row of dominoes. The practices I identified above, which are so often refuted as "adding complexity", are some of the many ways to guide your application toward this:
And away from this:
There is a multitude practices, patterns, and design principles that help keep your modules small, their scopes limited, and their boundaries well-defined. YAGNI is one of them, but not the only one. Claiming YAGNI to avoid this kind of work is
need it, right from day one. Working without these tools is seeding the ground of your project with the thorns and weeds of complexity. They provide you with a way to keep your code garden weed-free. In this way they are kin to YAGNI, not its enemy. Claiming otherwise reveals either a disrespect for, or a lack of understanding of, the benefits of good design and engineering practices in a general sense. So next time someone sets up this contradiction in front of you, don't let them get away with it. Show your knowledge, and stand up for quality and craft.
Any programmer who is patient enough to listen has heard me evangelizing the virtues of Test-Driven Design. That is, designing your application, your classes, your interface, for testability. Designing for testability unsurprisingly yields code which can very easily have tests hung onto it. But going beyond that, it drives your code to a better overall design. Put simply, this is because testing places the very same demands on your code as does incremental change.
You likely already have an opinion on whether that is correct or not. In which case, I'm either preaching to the choir, or to a brick wall. I'll let you decide which echo chamber you'd rather be in, but if you don't mind hanging out in the pro-testability room for a while, then read on.
Last week I began a new job. I joined a software development lab that follows an agile process, and places an emphasis on testability and continuous improvement. The lead architect on our development team has encouraged everyone to develop ideally in a test-first manner, but I'm not sure how many have taken him up on that challenge. I've always wondered how well it actually works in practice, and honestly, I've always been a bit skeptical of the benefits. So I decided this big change of environment was the perfect opportunity to give it a shot.
After a week of test-first development, here are the most significant observations:
Progress feels slower.
My classes have turned out smaller, and there are more of them.
My interfaces and public class surfaces are much simpler and more straightforward.
My test have turned out shorter and simpler, and there are more of them.
I spent a measurable amount of time debugging my tests, but a negligible amount of time debugging the subject classes.
I've never been so confident before that everything works as it is supposed to.
Let's break these out and look at them in detail.
1. Progress feels slower.
This is the thing I worried most about. Writing tests has always been an exercise in patience, in the past. Writing a test after writing the subject means sitting there and trying to think about all the ways that what you just wrote could break, and then writing tests for all of them. Each tests include varying amounts of setup, and dependency mocking. And mocking can be tough, even when your classes are designed with isolation in mind.
The reality this week is that yes, from day to day, hour to hour, I am writing less application code. But I am re-writing code less. I am fixing code less. I am redesigning code less. While I'm writing less code, it feels like each line that I do write is more impactful and more resilient. This leads very well into...
2. & 3. My classes have turned out smaller, and there are more of them.
My interfaces and public class surfaces are much simpler and more straightforward.
The next-biggest worry I had was that in service of testability, my classes would become anemic or insipid. I thought there was a chance that my classes would end up so puny and of so little presence and substance that it would actually become an impediment to understandability and evolution.
This seems reasonable, right? Spread your functionality too thin and it might just evaporate like a puddle in dry heat. Sprinkle your functionality across too many classes and it will become impossible to find the functionality you want.
In fact the classes didn't lose their presence. Rather I would say that their identities came into sharp and unmistakable focus. The clarity and simplicity of their public members and interfaces made it virtually impossible to misuse them, or to mistake whether their innards do what they claim to. This enhances the value and impact of the code that consumes it. Furthermore it makes test coverage remarkably achievable, which is something I always struggled with when working test-after. On that note...
4. My tests have turned out simpler, and there are more of them.
The simple surface areas and limited responsibilities of each class significantly impacted the nature of the tests that I am writing, compared to my test-after work. Whereas I used to spend many-fold more time "arranging" than "acting" and "asserting", the proportion of effort this step takes has dropped dramatically. Setting up and injecting mocks is still a non-trivial part of the job. But now this tends to require a lot less fiddling with arguments and callbacks. Of course an extra benefit of this is that the test are more readable, which means their intent is more readily apparent. And that is a crucial aspect of effective testing.
5. I spent a measurable amount of time debugging my tests, but a negligible amount of time debugging the subject classes
There's not too much to say here. It's pretty straightforward. The total amount of time I spent in the debugger and doing manual testing was greatly reduced. Most of my debugging was of the arrangement portions of tests. And most of that ended up being due to my own confusion about bits of the mocking API.
6. I've never been so confident before that everything works as it is supposed to.
This cannot be overstated. I've always been fairly confident in my ability to solve problems. But I've always had terrible anxiety when it came to backing up correctness in the face of bugs. I tend to be a big-picture thinker when it comes to development. I outline general structure, but before ironing out all the details of a given portion of the code, I'll move on to the interesting work of outlining other general structure.
Test-first development doesn't let me get away with putting off the details until there's nothing "fun" left. If I'm allowed to do that then by the time I come back to them I've usually forgotten what the details need to be. This has historically been a pretty big source of bugs for me. Far from the only source, but a significant one. Test-driven design keeps my whims in check, by ensuring that the details are right before moving on.
An Unexpected Development
The upshot of all this is that despite the fact that some of the things I feared ended up being partially true, the net impact was actually the opposite of what I was afraid it would be. In my first week of test-first development, my code has made a shift toward simpler, more modular, more replaceable, and more provably correct code. And I see no reason why these results shouldn't be repeatable, with some diligence and a bit of forethought in applying the philosophy to what might seem an incompatible problem space.
The most significant observation I made is that working like this feels different from the other work processes I've followed. It feels more deliberate, more pragmatic. It feels more like craft and less like hacking. It feels more like engineering. Software development will always have a strong art component. Most applied sciences do, whether people like to admit it or not. But this is the first time I've really felt like what I was doing went beyong just art plus experience plus discipline. This week, I feel like I moved closer toward that golden ideal of Software Engineering.
I'm starting a new job on March 7. As I wrapped up my time with the company where I've spent the past 3 years, I spent some time thinking about the decisions and responsibilities that I was given, and that I took on, while there. One of the reasons I was hired was to bring to the development team my more formal education and professional experience as a software developer. And one of the responsibilities I was allowed was to lead regular discussions with the other developers. The goal of these discussions was to help our team to become more professional in the way we developed software. Of course it was only a matter of time before someone on the team brought up coding standards.
I've had enough experience with coding standards to have developed some skepticism to the way that they are typically applied. It sounds good, in theory. Bring some consistency, order, and predictability to the source code produced by the group. But in my experience, the practice rarely delivers on the promise. It's easy to start a fight on this topic, of course. It tends to be a "religious" topic in that people have strong and entrenched opinions, and are rarely shaken from them regardless of argument. This is unfortunate, because I think there is a way to make them work. But it requires thinking about coding standards in a different kind of way.
The challenge of establishing standards, and the reason standards tend to be so fractious, is that "what works" must necessarily vary depending on the case. But very rarely does an organization who establishes coding standards allow for much variance in response to new problems or process friction. To figure out why this is, and how we can deliver on some of the promise of standards without resorting to brainwashing, let's take a closer look at the types coding standards we see in the wild.
In my experience, coding standards tend to fall into one of two broad categories. There are strategic coding standards, which are general and outcome-oriented. And there are tactical coding standards, which are specific and mechanics-oriented.
I'll address the latter first, as these are the kinds of standards that tend to start religious wars. Here are some examples of tactical coding standards:
All method headers must include a description, pre-conditions, side-effects, argument descriptions, and return value description.
Use C#'s "var" keyword only for variables of anonymous data types.
Place each method argument on its own line.
At this point you're probably at least nodding your head in recognition. Maybe some of you are nodding in satisfied approval, while others are holding back an outburst of disgust. When most people hear the words "coding standards", the tactical ones are the ones they think of. And these often evoke strong emotional responses. Their most common adherents tend to be managers or project leads, and often regular developers who've been around the block a few times.
The reason people want tactical coding standards is simple. They bring a modicum of consistency to a wild and woolly world. You often can't trust that the guy writing code in the next cube can code his way out of a paper bag. So when you have to take over his code after he's gone it's nice to know that, at the very least, he followed the standards. If all else fails, there is one thing that will provide sure footing as you plumb the dangerous depths of their code.
I understand this, and I sympathize. But at the same time, I chafe under these standards. They feel like micromanagement to me. It makes me feel like my coworkers don't trust me to do the thing that I was hired to do. There are organizations where this can't be taken for granted. But my own personal response to those environments is to leave them. I want to work with people I can trust at this level, and that can trust me the same.
It's not all bad for tactical standards, and we'll circle back in a bit. But for now let's look at some examples of strategic coding standards.
Write informative class headers.
Endeavor to keep methods to less than 25 lines in length.
Use whitespace and indenting to identify related code.
Strategic coding standards are broad. They do not tell the developer exactly what to write. Rather they set an expectation of the kind of code that should be written, and leave it to the developer as to how exactly that kind of code should be created. In my experience, these are the kinds of standards that are okay to mandate and enforce from a management perspective. Evoke the general feel of the code you want your team to produce, and then let them use their skill, experience, and professionalism to decide how to make it happen.
You can probably tell that in general I feel a lot better about this type of standard than the other. For one, they stay out of your way on a minute-to-minute basis. They're also far easier to get people to agree upon. Rather than being a cynical defensive measure against bad code, they empower the developer while fostering good coding habits.
Now having said all that, I do think there is a place for tactical standards. One big reason that I named these categories as I did is because I think that the role of establishing each type of standard can map to the organization similarly to how the roles map in the military, from which these terms are taken. Strategic standards can be mandated by leadership without stifling the developers. While the particular tactics can be determined by the people "on the ground".
There are many cases where tactical coding standards are beneficial. But it almost always serves the organization better to let the developers establish and enforce them. Take a team with multiple people all sharing equally in ownership of a codebase, each working from day to day on code written by the others. Imagine further that the code is sensitive in some way that can't abide regression bugs or the like. Maybe there is also a huge amount of code, or a large number of coders.
In these types of situations, it's crucial that the developers be able to pick up a strange piece of code and get a quick picture of what's going on. And to be able to modify the code without introducing a foreign-looking island of code to trip up the next guy. To do this, the developers have to be comfortable with the low-level style aspects of the code, both to read and to write. The best hope a team has of developing this comfort is to, as a self-directed group, come to agreement on common-denominator style standards over time and through shared experience.
There is a balance to be struck here. We want to ensure quality output from our teams, but we don't want to turn our developers into code-generators that don't take responsibility for their own work product. I think this balance is best struck by the proper application of each type of coding standard, in the proper context. If leaders can establish strategic standards that express high-level goals for clean code and understandable designs, then they can empower their teams to find solutions to the lower-granularity challenges such as understanding each other, and reducing the friction involved in cooperation on a shared codebase. This is strategy and tactics in practice, and at its best.
Deciding how to trigger the work of the service usually comes down to some sort of threading solution. And threading in Windows services is, as everywhere else, a thorny issue. The most basic need we have is to be able to respond to orders from the Service Control Manager (SCM) without blocking processing at an inconvenient spot. Beyond that, we may want to have a bit of work kicking off periodically, with a short period, or we may want to respond to some environmental event.
Responding to environmental events is fairly easy when the framework provides a callback mechanism for it, such as the FileSystemWatcher. But if there's no such facility, then you're stuck with polling, which is in the same boat as the guy who needs to do work on a quick periodic schedule. This leads us to worker threads, and all the headaches that go along with that.
I have no intention of getting deep into threading concepts and strategies here. Not least because I am woefully uninformed and inexperienced with them. If you need to be highly responsive, highly predictable, or highly parallel, that's an entirely new problem space. So it will have to suffice to say here that you should not just hack something together quickly. Do your research and do it right.
What I have done in the past is to use a Thread.Timer to fire a callback on my desired period. But I use a flag to make sure multiple work units do not process in parallel, and I don't worry about missing a period by a nanosecond simply because the work completed just after I checked the flag. Instead I just wait for the next one. This limits the responsiveness of my service, but it's simple and reliable because it sidesteps the issues that arise out of mutually writeable shared state and truly parallel work.
For our sample project here, I introduced this mechanism by creating a worker class that manages the timer. Let's take a look at that.
Note the flag properties, which are used as a guard on the timer event to make sure that processing doesn't re-enter before the previous iteration completes. The timer interval is set to start things rolling, and cleared to stop them. The added complexity in the Stop method is crucial to understand as well. Timers are tricky to get rid of without blocking the thread. We use a wait handle to do it. Wait handles are worth reading up on if you deal with threading at all. Another nice feature of this worker class is that an object of this type can be reused: Start can be called to spin things up again after Stop has finished.
The worker presents essentially the same interface as the service itself, which may seem a bit redundant. All I am really trying to achieve by this is to respect the separate responsibilities of management of service as a whole, as opposed to management of the work timer and triggers. The service itself is just processing events from the SCM. It need not know, when it tells the worker to start, whether the worker is firing up a new thread to get started, waiting for the moment the current one finishes, or just taking a pass on this period.
The service class will now delegate to the worker. This is very straightforward. I just had to replace the IGreeter property with a property for the worker. Now in OnStart, we spin up the worker, and in OnStop we finally have some code as well, to spin down the worker.
Now that we've got the service doing everything it's supposed to do, we may decide that we'd like to be able to change exactly how it does it's job. Essentially, we want to make it configurable. The only real parameters of our service at this point are the work interval and the file name, and those are hard-coded. We could easily make the service more flexible by taking these hardcoded values and making them configurable.
One way to do that would be to use the service argument array, which operates just like the regular app argument array. We don't have much to configure here, so that would probably work for us. But to imitate a real-world scenario where there may be lots to configure, let's pretend that won't be sufficient.
Another option would be to use a config file. Services, being normal .NET executables, can have app.config files just like a command line or Windows app does. I'm personally not fond of the default .NET config mechanism though, so I think I'll roll my own, because that can be quick and easy too, especially when our needs are as simple as they are here. Let's throw together a little XML format for our two settings.
We can load this pretty easily without relying on the magic of the .NET configuration mechanism, by creating a serializable XML data object. We can put it behind a general settings interface, and have a loader class to populate it. The interface will be enough to shield the rest of the app from the persistence details.
Here is the data class, and the settings interface.
The loader class is nearly as simple. Standard XML deserialization.
We can ensure that the code that uses these classes is kept simple, by registering these properly with the IoC container.
This registration will ensure that any IConfigSettings-typed dependency within a given Autofac lifetime context will receive the same ConfigSettings instance. The first resolution of that ConfigSettings instance will be created using the ConfigLoader service.
After identifying what we want to be configurable, we can go about defining some settings providers. I'll remind you that the value of settings providers is that they allow you to define dependencies on a given configuration value, while also decoupling the dependent class from the other unrelated settings that happen to cohabit the configuration source.
Below the work interval settings provider. I'll spare you the very similar code found in the file name settings provider. The IoC registration is likewise straightforward.
With the service now configurable, we can perform one final trick. When a service stops, the process may not necessarily stop. So we need some way to establish when the configuration is reloaded. The natural place for this to happen is when the OnStart event is triggered. Since the config file is loaded when a new IConfigSettings dependency is resolved for the first time in a context, we just need a new context to be spun up with each call to OnStart.
We can make this happen by taking advantage of Autofac's relationship types again. A Func<T> dependency will re-resolve each time the function is executed. And an Owned<T> dependency will cause a new lifetime scope to be created each time the dependency is resolved. So all we need to do is combine the two.
Here is the new service class, with the Func<Owned<T>> dependency.
When OnStart is called, assuming the service isn't already running, we create a new worker, from the delegate dependency. With this new worker comes a new scope, so it's dependency on IConfigSettings will be resolved with a new object, and the config file will be loaded on the spot. Conversely, OnStop allows the worker to go out of scope, resulting in the lifetime scope being cleaned up.
With that, we have finally completed our Hello World Windows service. The guts are all there, and it's cleanly designed around interfaces for maximum testability should we desire to have a unit test sweet. And finally, we use Autofac to conveniently manage a relevant object lifetime scope, making sure our settings are loaded and unloaded at the appropriate time. The actual core behavior of the service itself can be as simple or as complex as it needs to be, while remaining easily accessible and manageable via the worker object and it's lifetime scope.
The full source for the service in its final incarnation can, like all the waypoints, be found in this post's GitHub repo. Thanks for bearing with me through this month of Windows service architecture! I'll see if I can't find something new and different to tackle for next week.
After going through the remaining work I had planned for our service specimen, I realized that the resulting post would be very long, and so I have decided to extend the series by one more post. This week we give the app a job, and add logging.
We've spent enough time working on this service so far that we should probably give it a job at this point. You may have guessed from the name by this point that I've been intending to make this a "hello world" for services. The types of things that services typically do are either to respond to system events or network communications, or to process units of work as they become available. A nice easy job we can give to our service is to touch a file on a very short interval--an interval short enough that a scheduled task isn't appropriate. Let's say 5 seconds.
We'll worry about handling the interval later. For now, let's just add the behavior so that it happens once, when the service starts.
First we create a straightforward little task-oriented class and interface. This should require no explaining. And of course we'll register this with the IoC container as well. You should be familiar with that code by now, so from here on out, I won't mention the IoC container unless something out of the ordinary is required for the registration or resolution.
With this done, we add a dependency to the interface in our GreetService constructor, and then a call to the task in OnStart.
This will cause the C:\Hello\World.txt file to be created when the service starts. The service will then be running, according to the Service Control Manager (SCM), and can be stopped. But while running it will do nothing more. If you want to see the full sample project source at this point of progress, I have tagged it in the GitHub repo.
This is a good time to add some logging. If we have problems later on, logging will help us to troubleshoot what's going on. Right now, if the service were to fail somehow, we wouldn't know why. If AutoLog were set to true in the service contructor, then the service would log to the Application event log when it crashed, but not with much useful information. But we don't want that anyway, if our log is an important one. Rather it should have it's own log, so we can always be certain we're looking at the right info and we don't have to sift through hundreds of other entries to find it.
Adding a custom event log is actually quite simple. There are two parts to the task. One is to make sure it gets created when the service is installed. The other is to use it. Creating an event log on install is trivial, mostly because it's already being done, and we can just hook our own data onto that. That's right, the ServiceInstaller class by default contains an EventLogInstaller that is configured to install an event source for our service in the Application event log. All we need to do is override that with our own log and source name.
Since both the installer and the logger class we'll write will need to know those two pieces of text, we'll create a settings provider for them. This will allow us to control the values for both usages in a single place.
Now that we've got a source from which to obtain the proper names, we can add a dependency to our service installer, and use it to override the default event log settings. All we need to do is pull the default EventLogInstaller out of the ServiceInstaller.Installers collection, and change the properties.
When we go to install the service using InstallUtil we will see it output that it is creating a new event source and event log. And if you go to the Windows Event Viewer in the Administrative Tools, you'll find a log called GreeterService.
Of course in order for anything to show up in there, we'll need to log some things to it. The safest way to do that is to throw in a logger object that can try to log to the event logs, but which won't crash out if it can't. The last thing we want is for the service to crash due to a log issue if the work itself is humming along. (Unless, of course, we need an audit trail or something, but that's a different story.) Note the explicit exception suppression in the logger class below for this purpose.
This logger could be enhanced in a few ways, for more serious services. One good thing to do is to support different "levels" of logging. I have four standard levels that I use. In increasingly voluminous order they are: fatal errors, exception cases, progress, and verbose. An enum parameter added to the logging methods would allow the logger to check to see if it's in a mode that should allow an entry for the specified level. A step further, we might replace the String parameters with Func callbacks. The callers can place the actual work of assembling the message into these callbacks, and the logger would only call them if it had determined that the logging would be performed. By only doing the string manipulation if it is absolutely necessary this keeps performance up and avoids unnecessary failure points.
With the logger object available it would be a good idea to add some logging when the service is starting and stopping. Here are the updated OnStart and OnStop methods from our GreetService class:
Now you can install the service using InstallUtil.exe, and when you start and stop via the SCM, you'll see entries show up in the GreeterService event log.
We'll stop here for now, as this post is getting long and there is much more to do. You can see the full service code to this point so far where I have tagged it in the GitHub repo. Next time will be our final (I promise) post in this series. We'll bring it home by figuring out the best way to make sure the service does its work when we want it to, and then making it easily configurable, and reconfigurable.
