Taking IoC beyond DI with Autofac Part 2: Relationships
This week I want to talk a little more about lifetime control, and a lot more about categories of dependency, also known as relationship types. Nicholas Blumhardt, the principal author of Autofac, has a great blog post on what he calls “the relationship zoo” which I think goes a long way toward covering this space. I was originally going to do something similar, but his post is far more authoritative, and I’m quite certain I couldn’t do better. So instead, I'll abstain from the explanation and code samples and stick to analysis and rumination. So go read his post, then please do come back here and I’ll expand on it with some of my own thoughts.
…
The reality is that I take issue with some of the dependency abstractions provided by Autofac. It can be quite dangerous to your design to rely on some of them without very good reason. Used properly and prudently, they can absolutely address some particularly painful problems, especially when the dependencies consist of code you don't own and can't change. But it’s also easy to let them creep in wherever they appear to be convenient, and severely corrupt your design and architecture by doing so.
Let me start with the warnings, and then I’ll move on to extoll some virtues that I think Nicholas neglected to identify.
The first relationship type to be wary of is Lazy<T>. The intended purpose, as Nicholas explains, is to avoid expensive operations or construction until or unless it’s necessary. The idea of a lazy object is an old one. It’s a pattern that has been around for a while. My opposition to the usage of this relationship type is primarily that the need for laziness is usually a function not of the usage by the dependent class, but rather of the implementation of the module that is depended upon. Where possible, the consumer should be ignorant of implementation details of its dependencies. Let’s recognize that expensive operations are rarely truly transparent or ignorable, but the fact remains that the responsibility for this situation lies with the module being depended on, not on the consumer.
I strongly believe that if the code of the module that will be resolved for this dependency is under your control, then it behooves you to wrap the laziness around this functionality elsewhere... either building it into the implementation, or wrapping it with some sort of facade. Where Lazy<T> comes in handy is when you don’t have control over this code, and the class poorly encapsulates its functionality such that it can’t sufficiently be wrapped in a facade. At that point the consumer cannot pretend to ignore the situation, and may benefit from delegating the responsibility for the laziness to the container.
I’ll attach my second warning to the Func<T> relationship. I’m wary of the plain Func<T> because it overlaps a great deal with both Lazy<T>. It shares the same issues as Lazy<T> while adding the sin of looking suspiciously like a service locator. Service locators can be dangerous because they subvert the goal of inverting control. A service locator hands control back to the consumer by saying, “just call when you need X”, rather than handing off the dependency and saying “I know you need X so here, use this for that”. This is very rarely the appropriate way of expressing the relationship. The exception would be in the case that the consumer knows for certain that it will require multiple instances of the service it depends on.
Let’s spin things back to the positive by looking at Func
The IEnumerable<T> relationship is similar to this last scenario in that it offers a way to say “I depend on all implementations of T, no matter what they are”. This is probably a rarer scenario. Usually a service will have one key implementation, or a couple with very specific and differing purposes which will be used separately in different places. The most likely way for an inclusive but generalized need like this to arise is in an add-on or plug-in scenario. And in that case, you’re likely going to need some up-front processing to get things loaded up before the implementations can be passed off as dependencies to other objects.
I should note that it is probably not all that unusual for IEnumerable
One more small step along the path is Meta<T, M>. This is relationship type that specifies a dependency not only on a service of type T, but on some piece of metadata of type M about the module that will be provided at runtime. This metadata may be used to decide whether or not to go through with making use of the service, or how to do it. In fact, metadata is a great way to handle the special considerations a consuming object may need to make for a lazy service implementation involving a long-running operation. Maybe 90% of the time, the app can simply halt for an operation, but for certain implementations of the service, it’s more prudent to display a friendly “please wait” message and a progress bar. Attaching metadata at registration is a great way to enable these types of decisions intelligently without hard-coding a universal expectation one way or the other.
The final relationship type to address is Owned<T>. This one can be quite handy. At first blush it seems like this might violate the same principles as Lazy<T>, but if you think about it, they are actually quite different. Owned<T> indicates that the object expects to be solely responsible for the fate of the dependency passed to it. This tells the programmer, “I’m going to use this up, and it won’t be any good when I’m done with it, so just clean it up afterward.” Believe it or not, this fits in perfectly with the recommended implementation pattern for the .NET IDisposable interface. That is, that at some point, some object takes ownership of the resources, and responsibility for calling IDisposable, absolving all other transient handlers of the same. Ownership is sort of the dual, or inverse, of a constructor dependency. The constructor dependency says “I know I’m going to need this object”, and the ownership claim says “when I’m done with it, it can be tossed out.” And Autofac happily obliges, releasing the dependencies when the consumer itself is released, calling IDisposable.Dispose as appropriate.
After a twitter conversation with Nicholas himself, it became obvious to me that Owned<T> is probably the better solution to my qualms about InstancePerLifetimeScope. By establishing that the consumer “owns” its dependencies, we have essentially established exactly the limited and definite lifetime context that I was asking for! Behind the scenes, a lifetime scope is spun up when the object is instantiated, with nested resolution rules being applied as a matter of course. And when the object is released, then so is the lifetime scope and anything that was resolved as a new instance in that scope. However, we do have a symmetrical limitation here. Both the creation and the destruction of the context are tied to this object. Once the context is created, it can’t and/or shouldn’t be shared with anything either above or adjacent to this object in the dependency graph, or the deterministic disposal Autofac prides itself on will subverted and become unreliable.
In these past two posts, I’ve covered the whole continuum of dependency creation, lifecycle, and relationship strategies that go beyond simple Dependency Injection to fill out the breadth of what Inversion of Control really means. And they’re all available via Autofac to be handled (for the most part) separate from the implementation of the consumer, and without writing boilerplate factory classes or resource management classes to make it work. I hope that in reading these posts some people may see that IoC is a broad space of patterns and solutions, and that IoC containers are powerful and useful and far beyond the naive use and even abuse that people put them to for simple DI.
Taking IoC beyond DI with Autofac Part 1: Lifecycle Control
A Noob's Scattered Thoughts on REST
Clean Injection of Individual Settings Values
Community Service
- How do I do X?
- How do I do X in a clean, maintainable, and elegant way?
But I think it's crucially important to make a distinction between the two forms. The second form is a far more difficult question, asked far less commonly, and to which good answers are even rarer. When questions of this form are asked in forums, people are usually given a link to a page describing a pattern, or if they are really "lucky" a page with a sample implementation of that pattern. To be fair, patterns usually are the answer to these types of questions. But what we find written up on the web, or even in books, is most commonly pathetically oversimplified, without context, and often even without guidance on what support patterns are necessary to obtain the benefit or when alternatives may be preferable.
Essentially, most developers are left with no choice but to apply Matt's answer to form 1, to questions of form 2, in a much less information-rich environment. I contend that, while it may be one of those proverbial activities that "build character", it is ultimately more likely to be harmful to their immediate productivity--possibly even to their continued professional growth.
What we end up with is a pandemic of developers trying to hack out pattern implementations, being discouraged by the fact that the pattern seems to have no accounting for any possible deviations or complications in the form of the problem. Worse, developers are often dismayed to find that the one pattern they were told to use is merely the tip of a huge iceberg of support patterns without which the first may actually be more problematic than an ad hoc solution. Most often the developer in this position will end up determining that their deadlines will never allow them to go through the painful trial and error process on every one of these patterns, and accordingly drop back to the ad hoc solution.
It's time that we acknowledge that software development, whether you consider it an engineering discipline, an art, or a craft, has a history--albeit a short one. Things have been tried. Some work, some don't. There do exist "solved problems" in our problem space. To say that every developer should try and fail at all these efforts on their own ignores and devalues the collective experience of our community. Worse, it stunts the growth of the software development industry as a whole.
Yes, one can learn these things by trial and error. Yes, the understanding gained in this way is deeper and stronger than that gained initially by being tutored on how to apply the solution. And yes, there's a certain pride that comes with getting things done in this way. But this is not scalable. Putting each person forcibly through this crucible is not a sustainable strategy for creating experienced, productive, wise programmers. Every hour spent grappling in isolation with a question of "why won't this pattern do what I've been told it will" is an hour that could be spent creating functionality, or heaven forbid solving a new problem.
That is why those of us who have managed to obtain understanding of these "solved problems" must be willing to shoulder the responsibility of mentoring the less experienced. Being willing to explain, willing to discuss specifics, mutations, deviations, exceptions. Willing to discus process, mindset, methodology. These are the things that make the distinction between a programmer and a software developer, between a software developer and a software engineer.
The internet may very well not be the appropriate place to seek or provide this type of mentoring. I suspect it's not. And unfortunately, there are too many development teams out there buried in companies whose core competencies are not software, and consisting solely of these discouraged developers, lacking an experienced anchor, or even a compass. There are online communities that attempt to address the problem at least partially. ALT.NET is one such, for .NET technologies. But there really is no substitution for direct, personal mentorship.
So I would encourage young developers out there to seek out more experienced developers, and ask them the tough, form 2 questions. And I would even more strongly encourage experienced developers to keep a watchful eye for those in need of such guidance. Maybe even consider proactively forming a local group and seeking out recruits. Be willing, able, and happy to provide this guidance, because it benefits all of us. Every developer you aid is one less developer creating an ad hoc solution which you or I will be condemned to maintain, overhaul, or triage somewhere down the line.
Incidental Redundancy
A note: I originally composed this blog post in June 2008, but lost it in my backlog and it never got posted. In the interim, Robert C. "Uncle Bob" Martin has addressed the issue in his Clean Code Tip of the Week #1. He describes the issue and the dilemma far more concisely than I have here, and even provides a strategy for dealing with it. By all means feel free to skip my post here and consider him the "official source" on this topic. =)
I am a big fan of "lazy programming". By that of course I mean avoiding repetition and tedium wherever possible. I mean, that's what most programming is really about, right? You have a problem: some process that is annoying, unwieldly, or even impossible to perform with existing tools. And the solution is to write a program that takes care of all the undesirable parts of that process automatically, based on a minimized set of user input or configuration data.
The realities of programming on an actual computer rather than in an idealized theoretical environment rarely allow this ascending staircase of productivity to be climbed to the top step. But it is nevertheless a goal we should always strive for.
Hence Jeff Atwood's recent post about eliminating redundancies.
Anything that removes redundancy from our code should be aggressively pursuedAn honorable goal, to be sure. But no rule is without exception, right?
I humbly submit to you the concept of "incidental redundancy". Incidental redundancy is a repetition of code syntax or semantics that tempts the programmer to refactor, but if carried out the refactoring could damage the elegance and discoverability of the program.
The difference between incidental redundancy and regular redundancy in code is that the redundancy does not arise because of any substantive, or at least relevant, similarity between the two problems in question. Here are two ways I can think of off the top of my head for this to happen:
- The solutions you have come up with to this point for each situation just happen to have taken a similar form. Given a different creative whim, or even just an alternative refactoring, the commonality may never have arisen.
- The problems are, in fact, similar at some level. But the level at which they are similar is far above or below the level where you are working, and so not truly relevant or helpful to the immediate problems you are trying to solve.
The second situation may sound a little incredible. But allow me to point out an example from the world of physics. Please bear with me as this is an area of personal interest, and it really is the best example that comes to mind.
There are four forces in the known universe which govern all interactions of matter and energy, at least one of which I'm sure you've heard of: the electromagnetic, weak nuclear, strong nuclear, and gravitational forces. It is known now that the first two of those apparently very different forces are in fact two different aspects of the same phenomenon (the electroweak force), which only show up as different when ambient temperature is comparatively low. Most physicists are pretty sure that the third force is yet another aspect of that same phenomenon that splits off only at even higher temperatures. And it is suspected that gravity can be unified with the rest at temperatures higher still.
The point of all this is that there are four different phenomena which in fact bear undeniable similarities in certain respects, and these similarities continue to drive scientists to create a generalized theory that can explain it all. But no one in his right mind would try to design, for example, a complex electrical circuit based entirely on the generalized theory of the electroweak force.
The analogy to programming is that, were we to try to shift up to that higher level and formulate an abstraction to remove the redundancy, the effect on the problem at hand would be to make the solution unwieldly, or opaque, or verbose, or any number of other undesirable code smells. All at the cost of removing a little redundancy.
What we have in both physics and in our programming dilemma, is noise. We are straining our eyes in the dark to find patterns in the problem space, and our mind tells us we see shapes. For the moment they appear clear and inevitable, but in the fullness of time they will prove to have been illusions. But making things worse, the shadow you think you see may really exist, it's just not what you thought it was. That's the insidious nature of this noise: what you see may in fact be truth in a grander context. But in our immediate situation, it is irrelevant and problematic.
This concept is admittedly inspired heavily, though indirectly, by Raganwald's concept of "incidental complexity", where a solution takes a cumbersome form because of the act of projecting it upon the surface of a particular programming language, not unlike the way the picture from a digital projector becomes deformed if you point it at the corner of a room.
The real and serious implication of this is that, to put it in Raganwald's terms, if you refactor an incidental redundancy, the message your solution ends up sending to other programmers, and to yourself in the future, ceases to be useful in understanding the problem that is being solved. It starts sending a signal that there is a real and important correlation between the two things that you've just bound up in one generalization. When in fact, it's just chance. And so when people start to build on top of that generalization with those not quite correct assumptions, unnecessary complexities can quite easily creep in. And of course that inevitably impacts maintenance and further development.
Noise is ever-present in the world of programming, as in other creative and engineering disciplines. But it doesn't just come from the intrusive environment of our programming language or our tools as Raganwald pointed out. It can come from our own past experience, and even come from the problem itself.
So be wary. Don't submit unwittingly to the siren song of one more redundancy elimination. Think critically before you click that "refactor" button. Because eliminating an incidental redundancy is like buying those "as seen on tv" doodads to make your life "easier" in some way that was somehow never an issue before. You think it's going to streamline some portion of your life, like roasting garlic or cooking pastries. But in your quest to squeeze a few more drops of efficiency out of the tiny percentage of your time that you spend in these activities, you end up out some cash, and the proud owner of a toaster oven that won't cook anything flat, or yet another muffin pan, and a bunch of common cooking equipment that you probably already own.
So remember, redundancy should always be eliminated, except when it shouldn't. And when it shouldn't is when it is just noise bubbling up from your own mind, or from the problem space itself.
The First Rule of Extension Methods
Before you even begin to think about whether a particular behavior belongs on an existing class, or as a service, or what have you, you should internalize one cardinal rule of dealing with extension methods. Thankfully this First Rule of Extension Methods is nothing like the First Rule of Fight Club, because if it were, I wouldn't be able to help you at all. No, the First Rule of Extension Methods is: DO NOT export extension methods in the same namespace as your other public classes.
The reason for this is very very simple: odds are very good that if you have come up with an idea of including a particular new behavior on an existing class, someone else has or will as well. There will only be a finite number of relevant names to give this new method, which means that if it's a simple method, especially one without parameters, there's a decent chance that the signatures of your method and this other programmer's will be identical. And if both you and this other programmer happen to include the method on public static classes in a namespace with other functionality the user needs, then someone trying to use both libraries could very likely run up against identifier collisions with the two methods.
Note that the rule begins with "do not export..." This is important, because you can be 100% certain to save your users any collision headaches if you just don't export your extension method. Why wouldn't you export your extension method? Well, there's a very good chance that just because you found your extension method to be useful and, dare I say, clever (Tyler Durden: "how's that working out for you?"), that doesn't mean the consumer of your library will. So consider carefully whether you should even make the method public at all, or rather just keep it internal.
If you decide that your method is just so darned handy that to keep it locked up would just be sadistic, then make a separate namespace for each batch of related extension methods you want to export. If the user decides they'd like to use them too, they can import the namespace as necessary. This will minimize the chance that they will need to fully qualify the namespaces of the other library with the identically named method whose author wasn't as responsible as you were and polluted their namespaces with extension methods.
This tactic won't guarantee anything, because there's always a chance some other library could use the same namespace identifiers as well. But with each additional namespace, you dramatically increase possible naming permutiations, and proportionally decrease the chance of collisions.
Useful Extension Methods 1 through 3 of N
You may have also heard of this noise idea by another name: accidental complexity.
As Reg Braithwaite has pointed out, looping syntax is one of these things. With the ubiquity of IEnumerable and the advent of extension methods in C#, there is almost never a good reason to write an explicit for loop anymore. Looping is a ubiquitous bit of logic that nonetheless takes up quite a lot of characters. Even the vaunted foreach loop is now officially more verbose than it very often needs to be.
Microsoft made it easy to get rid of the explicit loop when what you are doing is essentially a mapping operation, with the inclusion of the IEnumerable
Say your Foo class has a static function taking a Bar and returning a Foo, and you want to use this function to take a collection of Bars and create a collection of Foos.
You could do this:
IEnumerable<Bar> bars = GetAllBars();
List<Foo> foos = new List<Foo>();
foreach (var bar in bars)
foos.Add(Foo.FromBar(bar));
Or you could do this, which is obviously much more concise:
IEnumerable<bar> bars = GetAllBars();
IEnumerable<foo> foos = bars.Select(Foo.FromBar).ToList();
Note that the Select function completely takes care of the looping logic. Once you know that, this code reveals itself as being extremely elegant. But what if the action you're taking doesn't return anything? You're "stuck" writing an explicit loop, right? Not at all.
Take this code:
IEnumerable<Foo> foos = bar.GetAllItems();
foreach (var foo in foos)
PrintToScreen(foo);
To start removing the noise, first define an extension method for IEnumerable
public static IEnumerable<X> ForEach<X>(this IEnumerable<X> lhs, Action<X> func)
{
foreach (X x in lhs)
{
func(x);
yield return x;
}
}
Then rewrite:
bar.GetAllItems()
.ForEach(PrintToScreen)
.ToList();
Now there's just the issue of that nasty little ToList call. Right now, we need that in order to force the collection to be iterated. The yield return syntax essentially causes a function's execution to be deferred until an element is actually requested. This is actually potentially useful even if none of the things you need to do will return values. You can chain together a bunch of actions on the collection by chain-calling ForEach with different delegates. But it's still silly to create and throw away a List just to do this.
So we create an Evaluate function that does a simple explicit iteration and nothing more, to force the iterator to be evaluated. This is extension method #2.
public static void Evaluate<X>(this IEnumerable<X> lhs)And now you can replace ToList with Evaluate, which will iterate the collection without allocating a new List.
{
foreach (X x in lhs) ;
}
bar.GetAllItems()
.ForEach(PrintToScreen)
.Evaluate();
This is nice, and is going to be useful if we need to chain ForEach calls. But when we don't need that there's still that Evaluate call at the end that's going to be repeated every time we want this functionality, which could be an awful lot. So, let's get rid of that too.
To do that we define a Visit function (named for the Visitor Pattern), that will call ForEach with the given delegate, and then Evaluate as well. This is extension method #3.
public static void Visit<X>(this IEnumerable<X> lhs, Action<X> func)
{
lhs
.ForEach(func)
.Evaluate();
}
Now we can finally get all this done in a single function call:
bar.GetAllItems()
.Visit(PrintToScreen);
This takes some getting used to. But it really has the potential to condense your code. It isn't readily apparent looking at one bit of code, but once you start talking about nested loops or consecutive loops, you'll see the difference. Not to mention the total effect it will have across your codebase. Loops are everywhere. Shave off 50 characters from each one and your talking about a lot of characters in aggregate.
For my part, after I determined to avoid explicit loops whenever possible, the comparatively verbose explicit looping syntax became almost painfully extraneous to my eyes. I feel like function calls are much more elegant.
Decoupling Domain Model from Persistence
I know this is true. I know these patterns exist. But as this is my first effort in developing a desktop app of this complexity, I don't really know what the patterns are. One that GoF did account for was MVC/MVP, and sure I know those... But honestly, up until recently (and maybe even still) my knowledge of that pattern was very hazy and academic. Actually implementing it was something I'd never had to do before. It took a lot of blood, sweat, and tears--lots of trial and error--to get to a point where I really feel like I'm starting to grasp the "right" way to put together an MVP structure.
So now I've moved on to persistence. I have my domain model. But I need to be able to persist the objects in my domain model to and from disk (not a database!). I've been struggling to find ways to add in this functionality without tainting the domain model with persistence logic. Granted, this is a valid way of doing things, as is evidenced by Martin Fowler's ActiveRecord pattern. But it really doesn't make for ease of unit testing. Your business logic and persistence logic get all mixed up in the same classes and you can't easily isolate one from the other for testing purposes.
Then I saw Fowler's Repository pattern. I thought, okay, maybe that will help. I can create an interface that I can implement which will take care of all the logic for disk access to all the right places, based on what objects or collections are being retrieved. And I can just mock the interface for purposes of testing the domain model and business logic. I have a simple three-level hierarchy of domain interfaces, so it would be fairly simple to implement all the querying and CRUD I need for each domain class on the repository interface. But then I thought about how we plan to have multiple implementations of these interfaces in the future, with different permutations of under-the-covers data to be persisted, and thought maybe it would be best to have the repository be metadata-driven, rather than hard-coding the query logic for the different domain classes. This of course naturally leads to a record-based design, and that would mean I'd want data mappers to translate from my domain classes to repository records. And of course, the repository class should be strictly independent of the persistence mechanism itself (in this case the disk) so that I can unit test the repository logic without worrying about maintaining a test repository just for that purpose.
And suddenly I realized that I had mentally re-created ADO .NET. DataSets and DataTables are the repository mechanism, (with typed datasets and the ADO designer even supporting generating data mappers for you). DataAdapters are the persistence service.
So where this leaves me is with the question of whether I should just design my on-disk storage schema, build myself an app-specific data adapter mechanism, and let ADO .NET do all the work. If I really wanted to minimize the amount of mapping and repository-access code I have to write, I might even be able to use the Entity Framework to do my dirty work for me. Admittedly I have no idea how much work it would take to implement my own data adapter. The interfaces seem straightforward enough, but I have a nagging feeling that's really just the tip of the iceberg.
After all my radio silence here, I don't know if anyone is still listening... But if anyone has any thoughts on the wisdom of this approach, I'm listening intently.
Don't Give Up Assembly Privacy For Sake of Unit Testing
Just a little PSA to other .NET unit testing newbs out there like me. This info is available various places on the internet, but you'll be lucky to find it without just the right search terms. So hopefully adding another blog post to the mix will make it easier to stumble on.
Unit testing frameworks need to instantiate your types, in order to run unit tests. What this means is that they need to be able to see them. The easiest way to do this is to make your classes public and/or place the tests right inside your project.
Placing the tests right in your project means that you'll very likely have to distribute the unit test framework assemblies along with your product. This might give more information to potential hackers than you would like. And making your classes public brings with it the often undesirable side-effect of opening up essentially all the types in your assembly to be used by anyone who knows where the assembly file is, in essentially any way they like.
But you don't have to make these concessions. You can move the tests out of your assembly, to keep them out of the deployment package, and still keep your classes internal (though not private). The .NET framework allows an assembly to declare "friend" assemblies that are allowed to see its internal classes. (Yes, very similar to the old C++ friend keyword). This is accomplished by adding an assembly attribute called InternalsVisibleTo to your AssemblyInfo.cs file.
If your unit test project does not have a strong name, it's as simple as referencing its assembly name:
However, I strongly recommend giving your unit test assembly a strong name. A strong name is a name involving a public-private key pair, and which is used by the .NET framework along with some hashing and encryption technology to prevent other people from creating assemblies that can masquerade as your own. Furthermore, if you give your app itself a strong name (which you should if you plan to distribute it), any libraries it references will need strong names, including the ones it just allows to see its internals.
So, if you decide to give your unit test project a strong name, you'll need the public key (not just the token) as well:
(If you need to learn about strong names, and/or how to extract the public key from your assembly, this is a good place to start: http://msdn.microsoft.com/en-us/library/wd40t7ad.aspx.)
Once you've done this, you should be able to compile and run your unit tests from a separate project or even solution, and still keep the classes you're testing from being "public".
This is all well and good, but if you're working with mocks at all, you probably have another problem on your hands. The most popular .NET mock frameworks (e.g. RhinoMocks, Moq, and NMock) use the Castle Project's DynamicProxy library to create proxies for your types on the fly at runtime. Unfortunately, this means that the Castle DynamicProxy library ALSO needs to be able to reference your internal types. So you might end up with an error message like this:
'DynamicProxyGenAssembly2, Version=0.0.0.0, Culture=neutral, PublicKeyToken=a621a9e7e5c32e69' is attempting to implement an inaccessible interface.
Complicating this fact is that the Castle DynamicProxy library places the proxies it generates into a temporary assembly, which you can't just run the strong name tool against, because the temporary assembly doesn't exist as a stand-alone file. Fortunately, there are programmatic ways of extracting this information, and the work has been done for us. The public key for this assembly, at the time of this writing, has been made available, here and here. You might find some code at those links that could help you extract the public key from any future releases of Castle as well.
The important information is basically that, as of today, you need to add this to your AssemblyInfo.cs file, without line breaks:
0004000001000100c547cac37abd99c8db225ef2f6c8a360
2f3b3606cc9891605d02baa56104f4cfc0734aa39b93bf78
52f7d9266654753cc297e7d2edfe0bac1cdcf9f717241550
e0a7b191195b7667bb4f64bcb8e2121380fd1d9d46ad2d92
d2d15605093924cceaf74c4861eff62abf69b9291ed0a340
e113be11e6a7d3113e92484cf7045cc7")]
Caution: The name and public key of this temporary assembly was different in earlier versions, and could change again in later versions, but at least now you know a little more about what to look for, should it change.
So remember: You don't have to open wide your assembly to just anyone who wants to reference your types, just for sake of unit testing. It takes a bit of work, but you don't need to compromise.
Surviving WinForms Databinding
I've rarely had the freedom in my career to implement a Windows application using MVC-style separation of concerns. Generally I am told "just get it working". Now, if I already knew how to tier out an application, this wouldn't be a problem. But since I don't, and it would take me a good deal of time to figure out a satisfactory way of doing it, I haven't been able to justify spending the time, on the company dime.
But recently, I've been fortunate enough to work on a new software product without a hard ship-date, and having other obligations on my plate. This has given me the freedom to not spend every minute producing functionality. So I've been experimenting with implementing MVC with Windows Forms.
There are essentially two ways to get this done. You can:
- Write a bunch of manual event code to trigger view changes from model changes, and vice versa.
- Use databinding and save yourself the explicit event code.
This works great if all your datasources are DataTables, or DataViews, or other such framework classes that are designed from the ground up to work with WinForms databinding. But should you have the misfortune of not dealing with any record-based data, you'll find you have a much tougher road to walk.
If you, like I, choose option 2, and you happen to be working on anything more complicated than a hello world application, and you are truly committed to doing MVC both correctly, and with as little unnecessary code as possible, then you will undoubtedly spend, like I have, a lot of time banging your head against a brick wall, trying to figure out why your databinding isn't doing what it is supposed to. This is a terrible shame, because if databinding in WinForms worked properly and was easy to use, it would be a spectacular tool for saving time and shrinking your codebase.
Truth is, you can still same time and code. But not as much as you might think when first introduced to the promise of databinding. If you can find any decent information on the obstacles you'll encounter. Compounding the above hurdles is the fact that what information can be found online about them is scattered to the four winds, and no one bit references the rest. So, I've decided to blog the headaches I encounter, and my resolutions, as I find them. This should increase the searchability of the issues at least a bit, by tying together the separate references with my blog as a link between them, and also by containing the different bits of information within one website. Or it would, if I had readers...
My results so far have produced 5 rules, to help you preserve your sanity while using WinForms databinding.
Rule 1: Use the Binding.Format and Binding.Parse events.
This first rule isn't actually too hard to find information on. Format and Parse essentially let you bind to a datasource property that doesn't have the same data type as the property on the control. So you can bind a Currency object to a TextBox.Text property, for example.
The Format event will let you convert from your datasource property type to your control property type, and Parse will let you convert from your control property type to your datasource property type. The MSDN examples at the links above are pretty good. If you use them as a template, you won't go wrong. But if you start to switch things up, beware Rule 2...
Rule 2: If you use Format or Parse events, DO NOT add the Binding to the control till after register with the events.
I honestly don't know what the deal is with this one. I just know that if you add your events to your Binding object after you've already passed it to the Control.DataBindings.Add function, they won't get called. I don't know why this should be, unless the control only gets a copy of your actual Binding object, not a reference to it.
Unfortunately, I have lost the references I had to the forum posts that talked about this. There were several, and now I can find none of them. I know I saw them, though, and I saw the symptoms of the other ordering, so as for me, I'm going to make sure to follow this rule.
Rule 3: Use INotifyPropertyChanged and/or INotifyPropertyChanging.
I ran across this info in a post on Rick Strahl's blog. I ran across this information during a desparate scramble to find out why my datasource ceased to be updated by user actions on the controls, after the initial binding occurred. The INotifyPropertyChanged interface is intended to be implemented by a datasource class that has properties that will be bound to. It provides a public event called PropertyChanged, which is registered with the data binding mechanism when you add the Binding to your Control. Your class then calls this event delegate in the desired property setters, after the new property value has been set. Make sure to pass "this" as the sender, and the name of the property in the event arguments object. Notice that the property name is provided as a String, which means that there is reflection involved. This will become relevant in Rule 4. Also note that there is an INotifyPropertyChanging interface, which exposes an event that is meant to be raised immediately before you apply the datasource change. This is generally less useful for databinding, but I include it here to save some poor soul the type of frustration I have recently endured.
Rule 4: If you implement INotifyPropertyChanged, don't include any explicit property change events ending with "Changed".
As I mentioned, the databinding mechanism uses reflection. And in so doing, it manages to outsmart itself. There is a very good chance you're going to run into a situation for which these databinding mechanisms aren't useful, and you'll have to implement your own explicit property change events on your datasource class. And of course, you're going to name these events in the style of "NameChanged", "AddressChanged", "HairColorChanged", etc. However, the binding mechanism things it's smart, and rather than just registering the INotifyPropertyChanged.PropertyChanged method, it will also register with any public event whose name ends with "Changed". And if you didn't happen to make your event follow the standard framework event signature pattern--that is, void delegate(Object sender, EventArgs e)--then you will get errors when the initial binding is attempted, as the mechanism attempts to register it's own standard-style delegates with your custom events, and you get a casting error.
I solved this one by following a crazy whim, but I also tried to verify the information online. All I could find was one old post buried in an obscure forum somewhere.
Rule 5: Don't bind to clickable Radio Buttons
I know how great it would be if you could just bind your bunch of radio buttons to an enum property. I really do. You think you're just going to hook up some Format and Parse events to translate back to your enum, and all will be well. It would be so darn convenient, if it actually worked. But WinForms just isn't cut out for this. For 3 full releases now (or is it 3.5 releases?), this has been the case. It's because of the event order, which is not something that MS can go switching up without causing thousands of developers to get really cheesed off.
The problem really comes down to the fact that unlike other controls' data properties, the Checked property of a radio button doesn't actually change until focus leaves the radio button. And as with all WinForms controls the focus doesn't actually leave the radio button until after focus is given to another control, and in fact not until after the Click event of the newly focused control has fired. The result of this, as it pertains to radio buttons, is that if you try to bind to them, the bound properties in your datasource will actually lag your radio buttons' visual state by one click. If you have just two radio buttons, the datasource will be exactly opposite the visible state, until you click somewhere else that doesn't trigger an action that references those datasource properties. Which can make this a really infuriating bug to track down. I almost thought I was hallucinating.
Now, in all honesty, it's possible to make it work. But it is the kludgiest kludge that ever kludged. Okay maybe it's not that bad... but it's a messy hack for sure. It takes a lot of work for something that really should already be available. As near as I can tell, the only way to solve this problem without giving up the databinding mechanism is to essentially make your own RadioButton control, with a property change and event order that is actually useful. You can either write one from scratch, or sub-class RadioButton and override all the event logic with custom message handling.
So....
There's the result of 3 weeks of frustration. I hope it helps someone else out there someday. I'll make list addendum posts if/when I come across any other mind-boggling flaws in the WinForms databinding model. And in the meantime, I welcome any additions or corrections that anyone is willing to contribute in the comments.
Is There a Place for Deletionism in Wikipedia?
Yesterday evening and this morning I watched (Or overheard? No idea what the appropriate term here is...) an exchange over Twitter between Tim Bray and Jeff Atwood about "deletionism". Tim's thoughts on the matter were apparently too strong to be held within the restrictive confines of Twitter and so he wrote a strongly worded post on his blog as well.
The post is obviously... passionate. But putting aside the bulk of the post that simply expresses that passion, he makes a couple really strong points.
The first point is that deletionists are just your garden variety elitists.
"the arguments from the deletionists are jargon-laden (hint: real experts use language that the people they’re talking to can understand)"
The other point really could have comprised the whole post and, IMHO, would have been a sufficient argument all by itself.
"What harm would ensue were Wikipedia to contain an accurate if slightly boring entry on someone who was just an ordinary person and entirely fame-free? Well, Wikipedia’s “encyclopedia-ness” might be impaired... but I thought the purpose of Wikipedia was to serve the Net’s users, not worry about how closely it adheres to the traditional frameworks of the reference publishing industry?" [emphasis added]
This, to me, undermines the deletionists' whole platform. That is what Wikipedia was supposed to be: a new model of information archival. A model not subject to the sensibilities of some authoritarian arbiters of what is "notable" or "interesting". And the deletionists are unequivocally trying to undermine this goal, by deciding what "deserves" to be archived.
I say, if they want to take that old dead-tree encyclopedia model and just port it to the web, they can go find their own site, and their own content, to do it with. I've even got a couple recommendations for them.
Identity Crisis in Computer Science Education
I knew what my personal complaint was, and I knew that it was connected to a larger problem with the state of computer science education in general. But I didn't know exactly what the larger problem was, let alone have any ideas worth sharing on what might be done about it.
Now that seemingly the entire remainder of the blogosphere has weighed in on this and related topics, I think I am finally ready to throw my two cents in. I'm going to barrage you with links in the following paragraphs, to ensure that I credit everyone I read who assisted me in coming to my final conclusions. Feel free not to click through, but be aware that they represent a rich cross-section of an important discussion.
It took me several weeks to realize that what we have going on is essentially a three-way tug of war from people in different regions of the vast sphere of software development, who need very different things out of their workers, and hence out of their education. Below I will give a run-down of some of the claims made, expressing the different forces pulling on CS graduates these days. You'll quickly see it's no wonder that the schools are so confused....
The Artisan Programmer
Joel laments the uselessness of theory courses in computer science curricula, saying "I remember the exact moment I vowed never to go to graduate school" and then proceeding to recall a terrible experience he had with a Dynamic Logic class. Jeff Atwood insists that real-world development environments need to be in place and mandatory, including, but not limited to, source control, bug tracking, deployment, and user feedback. Then, as mentioned above, Joel proposes offering BFAs in software development, to make darn well sure that none of the academic (in the pejorative sense) theory stuff gets mixed in unnecessarily. The upshot of most of these points are that computer science / programming degrees should spend as much time as possible teaching people what they need to know to go into a career in software development, writing business software or software products.
The Computer Scientist
Brian Hurt comes in from another direction entirely, and in the process makes some very good points about the true purpose of higher education. He lays the blame for the flood of single-language programmers entering the workforce at the feet of schools who do just exactly what Joel and Jeff are asking for. He makes some great points. And while he sounds more than a little reminiscent of the classic Joel post about the perils of java schools, his argument is much more thorough than just blaming the tools. Chris Cummer joins this party, wishing that his theory foundations had been firmer, and making an excellent analogy to the difference between someone who studies a language, and someone who studies language. We also have the respectable Raganwald, who although he has admirably pointed out good points from all sides, doesn't shy from offering his opinion that programmers ignore CS fundamentals at risk of their own career advancement.
The Software Engineer
But thats not all. Several people have weighed in from yet another direction. Robert Dewar and Edmond Schonberg wrote one of the posts that started off this blog firestorm. Along with alluding to a similar sentiment as Hurt and Cummer, they heavily criticize the state of software engineering education for focusing too much on how to use specific, limited tools, when there are more sophisticated ones available. They claim understanding these will allow software engineers to easily pick up the other tools that may come along and direct them to appropriate purposes. Ravi Mohan stops short of calling the education satisfactory, instead sensibly pointing out simply that an engineer who doesn't use standard engineering tools such as system modeling, isn't really an engineer. Mohan comes on a little too strong for me in the comments, but the posts themselves (of which there are also a precursor and a successor, and should soon be one more) are worth reading. Like the others he makes valid points.
Resolving the Crisis
Mark Guzdial is one of the few people that really puts his finger near the pressure point. Though maybe not near enough to feel the pulse beneath it when he did so. At risk of quoting a little too heavily....
Rarely, and certainly not until the upper division courses, do we emphasize creativity and novel problem-solving techniques. That meshes with good engineering practice. That does not necessarily mesh with good science practice.And there it is.... Different environments require different mindsets/approaches/philosophies. Research requires one mindset/philosophy of work, engineering requires another, and in-the-trench-based programming requires yet a third.
Computer scientists do not need to write good, clean code. Science is about critical and creative thinking. Have you ever read the actual source code for great programs like Sketchpad, or Eliza, or Smalltalk, or APL 360? The code that I have seen produced by computational scientists and engineers tends to be short, without comments, and is hard to read. In general, code that is about great ideas is not typically neat and clean. Instead, the code for the great programs and for solving scientific problems is brilliant. Coders for software engineers need to write factory-quality software. Brilliant code can be factory-quality. It does not have to be though. Those are independent factors.
When a person suffers from a personality fracture, the resolution is often to merge the personalities by validating each as part of a whole. Fortunately, since we are not dealing with a person, we have the freedom to go another direction: make the split real and permanent.
Associate of Science in Computer Programming
To fill Joel and Jeff's need, the student who wants to work in the craft of software development / computer programming, who wants to be an artisan, needs to have an appropriate degree. It needs to provide them with an exposure to the generalized idea of the programming platforms and tools that they will have to deal with for the rest of their career. Lose the lambda calculus, compiler-writing projects, etc. These things not necessary for them to get stuff done in the trenches. But they do need to be exposed to the fundamental generalities that pervade programming. And they need to be prepared to learn at an accelerated rate while in the field. That just comes with the territory. Focus on core programming skills like program analysis, debugging, and test practices. Introduce industry-standard tools (emphasizing generality and platform-independence) such as source-control, bug tracking, etc.
I think a two-year associate degree is perfect for the code-monkeys and business programmers that just love to dig in and mess around with code, and don't want to concern themselves with the overarching concerns. Especially with these jobs increasingly being pushed offshore, computer science grads are rapidly being priced out of the market. An associate degree is cheap enough to be worth the investment for a lower-paying programming job. And it doesn't carry the overhead of any unnecessary theoretical content that they may not be interested in learning. It should be noted though that this type of programming job enters the realm of the trades, with all the associated benefits and drawbacks.
If you're looking for a more well-rounded individual capable of moving up out of this position into a lead position, or even management, then a 4-year bachelor of science (or Joel's BFA, but I tend not to think so) may be a viable option as well.
Bachelor of Science in Computer Science
There's not much to say about this degree, because if you look at all the schools that are famous for their CS degrees, this is pretty much what you'll find. Lighter on general studies, heavy on theory, heavy on math. Light on tools because the students will be expected to find (or make) tools that work for them. Light on specific language education because students will be expected to adapt to whatever language is necessary for their problem domain.
This is a degree that will produce people primed for going on to masters and doctorates. They will end up in research, "disruptive" startups, or working on new languages, OSes, etc. This degree is designed for people who want to work at the edge of things. Who want to solve new problems and push the boundaries. They are people upon whom will be placed the burden of pushing the state of knowledge in CS into the next era.
Bachelor of Science in Software Engineering
I am hesitant to propose this degree, because I am not certain that the practice of Software Engineering has evolved to the point where we have 4 years worth of general knowledge that's worth teaching, and that won't be out of style by the time the student's graduate.
It seems that some people, when they talk about Software Engineering, are talking about architecture and design, and others are talking about process, resource allocation, estimation, etc. To be frank, I don't think the former qualifies as a true engineering discipline. At least not yet. I don't know how much the modeling of programs that Ravi Mohan talks about is going on out there in the industry. I suspect that it happens more in the process of really big projects, and maybe in digital security. The second type of engineering people think of, however, I think is very similar to what we see in manufacturing, with industrial and process engineers. These are people who get an intimate knowledge of the domain, and then figure out ways to get everything to run smoother, more efficiently, and producing higher quality.
I can definitely see some education possibilities here, though I am not sure myself how to fill out the whole degree. It should at least encompass a good portion of the Associate of Science in Computer Programming, because they need to understand the intricacies involved. I can also see this degree teaching some of the more established measurement and estimation techniques found among the industry's established and experienced software project managers. Generally more management-related topics such as resource allocation, planning, product design, feature negotiation, etc. might fit in well here. Different project processes, testing/QA models, and of course an ability to keep up to date with technologies and platforms, are all par for the course as it's all critical for making decisions in the industry.
Conclusion
I really, honestly believe that Computer Science education as a whole needs a makeover. It needs more structure, more integrity in the vision of what each degree means, across schools. When someone has one of these degrees, you need to be able to reliably assume they should have learned certain things, regardless what school they went to. Many of the degrees currently on offer don't satisfactorily prepare their students for any one of the possible careers discussed above. I'm not saying their hand needs to be held from enrollment right on through to their first job. That's not the purpose of college. The purpose of college is to provide a cohesive education, directed to some relatively well-defined goal of capability and knowledge. Today this is tragically non-uniform at best, and absent altogether at worst.
So I see plenty of room in software development education for a clarification of purpose, and a readjustment of goals and curricula. A few different tracks, each geared toward a distinct section of the sphere with different goals and different responsibilities. And if we resolve to use existing terminology with some respect for the historical meaning of the words, we can re-use our existing nomenclature. But there can be no more of this muddy slurry of computer science, craft of programming, and software engineering all overlapping in claims of purpose, treading on each others' territory without care. Everyone can have what they are asking for. They just need to accept that no one can claim the "one true way".
I am not a Computer Scientist
Prepare yourselves. I have an embarrassing and melodramatic admission to make.
My career is a sham.
Although my degree and education are in a field that is typically referred to as "computer science". I am not actually a "scientist". Nor do I "practice science". But I won't be satisfied to go down alone for this charade. I'll going on record saying that I am convinced that for the vast majority of people who were educated in or work in the field of "computer science", the ubiquitous presence of the word "science" in proximity to our work or education, is a tragic misnomer.
I don't know how long this has been on my mind, but I know almost precisely when I became conscious of it. It was a couple months ago. I was newly exposed to devlicio.us, and perusing the blogs hosted there, when I came across a post by Bill McCafferty about a lack of respect and discipline in our field.
Early in the post, Bill reveals an injustice he encountered during his education.
...When I started my undergrad in this subject, I recall reading articles debating whether it should be called a science at all. Gladly, I do not see this argument thrown around much anymore.
I think I am probably not going to make the exact argument here that he disagreed with back then. The things we all studied in school are definitely part of a nebulous field of study that may rightfully be called "computer science". As Bill points out,
"From Knuth's classic work in The Art of Computer Programming to the wide-spread use of pure mathematics in describing algorithmic approaches, computer science has the proper foundations to join other respected sciences such as physics, concrete mathematics, and engineering. Like other sciences, computer science demands of its participants a high level of respect and pursuit of knowledge."
I have no argument with any of this. He's right on. Donald Knuth (who is indeed my homeboy in the sense that we share our hometown) studied and practiced computer science (which if you know anything about Knuth, you'll know is an almost tragic understatement). And thousands of people who have followed in Knuth's foot steps can lay the same claim. However, that's not me. And it's not more than 99% of all programmers in the field today.
Computer science suffers the same type of misnomer as many other disciplines who have adopted the word "science" into their name, such as political science, social science, animal science, food science, etc. And it seems that most such fields, if not all, have done so because the very validity of the field of study itself was subject to severe criticism at some point in the past. So we take on the term "science" to get it through people's heads that there is a root in formal practices and honest intellectual exploration. But to then blanket every profession that derives from this root with the term "science" is a misappropriation of the term.
I can think of a number of examples.... The programmer working for the bank to develop their website, or for the manufacturing company to manage their transaction processing system is no more necessarily a "computer scientist" than the election commentator is necessarily a "political scientist". When someone gets an electrical engineering degree and goes to design circuits for a living we do not say he "works in electrical science". We say he is an electrical engineer. When someone gets a technical degree in mechanics and then goes to support or produce custom machinery, we do not say he "works in mechanical science". We say he is a mechanic, or a technician. Why, then, when someone gets an education that amounts to a "programming degree", and then goes to work doing programming, do we say that he "works in computer science"? It's a uselessly vague and largely inappropriate label.
By contrast, if you have a doctorate in computer science, I'm prepared to say you deserve the label. If you write essays, papers, articles, books, etc. for use by the general practitioner, you probably deserve the label. If you do research, or work on the unexplored fringes of the field--if you are exploring the substance and nature of the information or practices that the rest of us simply consume and implement, then in all likelihood you deserve the label.
Please, please understand that I am by no means belittling the value of our work, or the nobility of our profession. Often we simply consume the information produced by true "computer scientists". But we transform it from theory into practice. We resolve the concrete instances of the abstract problems that the true scientists formally define. We take the pure thought-stuff produced by scientists and turn it into tangible benefit.
This is not trivial. It is not easy. It deserves respect, discipline, study, and care. But it is not "practicing science".
I should say in closing that I am not as upset about all this as the tone of this post might imply. I don't even really have a big problem with the use of the word "science" to refer to a field of study or work that largely does not include research-type activities. I don't like it, but I accept that it happens. But "computer science" has a problem that other similar "sciences" don't. When someone says they work in "political science" or "food science", you can make a guess as to the type of work they do, and it's hard to be significantly incorrect. Though maybe it's my outsider's naïveté that allows me to make this claim. At any rate, "computer science" as a field is so broad and vague that I don't think the term communicates a useful amount of information. But you wouldn't know that by talking to programmers, who seem only too ready to attempt to take hold of the term and own it for themselves.
I think this is one small facet of a larger and far more critical issue in our field in general, which I fully intend to write more about very soon. But until then, lets take the small step of starting to consider what we really mean when we use much of the popular but often ambiguous terminology when discussing our profession.
I work in the field of computer science. This tells you nothing except that I am unlikely to be a prime specimen of the wondrous human physiology. But.... I am a programmer. I have a degree in Computer Engineering. I am interested in programming theory. I work as a software development consultant. And now, you know something about what I know and what I do.
Now what about you?
Update: I forgot to note that in McCafferty's blog entry, he himself makes use of "trade" terminology to categorize different levels of reading materials. Which belies the uncertain nature of programming as a profession. We certainly wouldn't say that a carpenter works in the "wood sciences", would we?
What is a Senior Programmer?
My first mistake was that, as I began my internship, I had no idea how my career would progress, how it should progress, or how active a participant I would be in whatever progression did occur. I knew that I enjoyed twiddling around with code, and that I seemed to have more of a natural talent for it than most of my classmates. And I figured that if I was going to make a career out of anything, it should be something that I did at least passingly well, and that I enjoyed.
As I entered my first post-college job, I had decided I most definitely did not want to become a manager. I enjoyed programming too much to give it up, for one thing. Further, managers had so far stood mostly as obstacles to my involvement in interesting work, and as incriminating figures more prepared to remonstrate me for my professional flaws rather than empower me to better myself as a programmer. So I wanted nothing to do with that. Instead I set a near-term goal of becoming a "senior developer", at which point I would re-evaluate my career trajectory and adjust course if necessary.
An important question in gauging the advancement of your career as a programmer is to ask what exactly it means to be a senior programmer. I have come to see this title as being tied to the professional respect that one has accumulated in a programming career. A senior programmer is not someone who has served a certain amount of time "in the trenches". Nor is it even someone with a broad experience base.
No, I think there is something a bit more intangible, that identifies someone deserving of the "senior developer" title. Something less measurable. Something that is probably sensed, but not necessarily explicable by those with less experience. But something that would be conspicuous by its absence.
As I see it, what distinguishes someone deserving of the "senior developer" title is a sense of stability. A senior developer is someone who can stand in the middle of the chaos that arises in a project, and exert a calming influence on the people and efforts swirling around him. A senior developer is an anchor to which a project can be tied to keep it from drifting into dangerous waters. He is a sounding board against which claims will ring true or false and goals will ring possible or impossible. He is the steady hand, not necessarily on the rudder of your project's ship, but wherever that hand is needed most. And he is a strong voice that reliably calls out your bearing relative to your destination.
Of course, these metaphors sound a bit grandiose. But the general picture I think is accurate. A senior developer is someone that you put on a project to ensure there's some measure of certainty in the mix. It doesn't mean your project is guaranteed to succeed. But it should mean that you can sleep a little easier knowing that where you think the project is, is where it really is. And that if it's not where it needs to be, that there is someone involved who has a decent idea of how to get it there.
Naturally, these things do come with time and experience. So what I said earlier isn't completely true, in that a senior developer is someone with tenure and experience. However, these are necessary, but not sufficient conditions. Not everyone with 20 years of experience on 5 platforms and 15 languages qualifies. And not everyone with 5 years of experience on 1 platform and 2 languages doesn't qualify. Rather, if you show that you can learn from experiences both positive and negative, port technical and non-technical knowledge from one domain to another, and educate, inspire, or empower colleagues and junior developers.... Then you are showing yourself to have what it takes.
If you're like me, and Nate, you don't feel that you're there yet, but you hope one day to proudly contribute this kind of value. Don't lose hope. Every new experience, technical and non-technical, is a growth opportunity. But it is important to broaden your horizons in both of those respects. If you hope to educate, inspire, and empower others, you must first learn to do so for yourself. And if there's one thing I've learned, it's that you can't do that if you feel stagnant. In that case, your first responsibility to yourself is to educate your boss of the professional value you could offer with a little broader exposure. And if that doesn't work, address the issue yourself, dedicating a bit of time outside work. If you can find them, working together with some like-minded friends or co-workers can be very encouraging, like what we have done with our "book club". Remember, nothing changes if things just stay the same.
LINQ: Not just for queries anymore
For those of you who don't know what the heck LINQ is, it was created as a language extension for .NET to allow querying of datasets using an in-place SQL-inspired notation. This was a huge step up from maintaining separate bodies of stored queries, or hard-coding and dynamically assembling queries as strings in your source code.
Lately LINQ has been enhanced and expanded upon even further to become a more broadly usable miniature functional/declarative language within .NET. This is impressively illustrated by the raytracer code. The author disclaims that it's probably not the best way to write it, which is probably true. But this in no way detracts from its illustration of the power, expressiveness, and flexibility of LINQ.
I love it! It's a great example of taking a deceptively simple language and showing its power by doing things with it that aren't strictly within the purview of its design. It reminds me of some SQL code that I've written for my current employer, to get functionality out of inefficient PL/SQL and into blazing-fast pure SQL. Stuff that it was said couldn't be done in pure SQL. Of course, this is usually said by people who think that SQL is a second-class citizen among languages, not realizing the power of its declarative foundations. This is something that I hope to write about more extensively in the future, either here or in a new SQL blog I'm considering starting with a friend and coworker.
Anyway, I'm tempted to say that there's a bit of a down side in that this LINQ raytracer is mostly comprised of LETs, which feel more imperative than declarative. However, I've long claimed that just such functionality in actual SQL would make it a tremendously more compact and elegant language, so I won't complain too much about that. =)
My gut reaction upon seeing this code is that it feels like a crazy hybrid of LISP and SQL, but it's written in C#. Which all makes my head spin, but in a good way. I love that C# is becoming such a great hybrid of procedural and functional programming paradigms.
Where will your programming job be in 7 years?
But now the noise is coming from some different directions. It's not analysts, or people who've been layed off, or the people who got screwed in the "dot.com bust". Though to be fair, the analysts are still saying it, louder than ever. But it's also now coming from the people who hire programmers. And unfortunately, if you are a programmer, those are the people you care about. So, what's different this time? Why don't companies need programmers anymore?
Well, they do need programmers. But they need programmers who can do more than just program. So odds are good that if your job title is "programmer", or "programming" constitutes 90% or better of your responsibilities on the job, they're looking at you like day-old tuna salad.
From the ComputerWorld article:
"It's not that you don't need technical skills, but there's much more of a need for the business skills, the more rounded skills""Crap," you say to yourself. "I hate business stuff." Or maybe you're saying "Crap. Why didn't I go for that business minor or second major back in school?" Easy, cougar. Don't get too worked up yet. Let's look at the context of this statement. We already know who's saying it: "business people". People working in IT in companies throughout the nation. Not software companies. Not consulting companies. Just regular companies. This is crucial information. It means the positions we are talking about are mostly going to be be "business programming" jobs.
Now we have a few more questions. Where are the jobs going? Why did I put quotes around "business programming"? And why are these jobs going away for real this time?
First answer: they're going to go to people with Associate degrees, and people who are self-taught, and consultants. Some to high-quality on-shore resources, but an awful lot to people who speak Hindi or Urdu (or Chinese, or Korean). Wages and education requirements will be cut for in-house employees, and others jobs are going to consultants. Offshore in most cases.
Next... The reason I put quotes around "business programming" is to distinguish these jobs from other types of programming positions. If you are a "business programmer", you're part of what is currently a pretty huge job market. You're part of a population that does a lot of work for a lot of companies, but is often looked at as an unfortunate necessity by other people at these companies. "Business programming" is the type of programming for which, in the past, a company who sells shoes, would hire some people on full-time and in-house to do. It's order processing, financial reporting, CRM, network management, database scripting, and so on and so forth. And for a long time, management types have had an uneasy feeling that they were getting kind of a raw deal on these guys. And I hate to break it to any business programmers out there.... but they were right.
According to the Occupational Outlook Handbook entry for computer programmers, there are very specific and very real reasons that programming jobs are being phased out.
"Sophisticated computer software now has the capability to write basic code, eliminating the need for many programmers to do this routine work. The consolidation and centralization of systems and applications, developments in packaged software, advances in programming languages and tools, and the growing ability of users to design, write, and implement more of their own programs mean that more of the programming functions can be transferred from programmers to other types of information workers, such as computer software engineers.I would add to this list that "business programming" is almost inherently redundant. Every company out there that employs in-house developers is reinventing the wheel. 99% of the problems their programmers are solving have been solved by thousands of other programmers at other companies around the country. When I look at it that way, it feels like such a tremendous waste of money and time. These programmers could be working on real problems like true AI, building Skynet, or bringing about the rise of the machines and their subsequent domination over the human race.
Another factor limiting growth in employment is the outsourcing of these jobs to other countries. Computer programmers can perform their job function from anywhere in the world and can digitally transmit their programs to any location via e-mail. Programmers are at a much higher risk of having their jobs outsourced abroad than are workers involved in more complex and sophisticated information technology functions, such as software engineering, because computer programming has become an international language, requiring little localized or specialized knowledge. Additionally, the work of computer programmers can be routinized, once knowledge of a particular programming language is mastered."
So essentially, the big reason is that there is finally a way for a company to separate their technical needs from their business needs. Packaged software has finally come to a point where it solves the general problems, while still providing a minimum amount of flexibility necessary to handle a company's critical peculiarities. When they have a need that isn't fulfilled by some packaged solution out there, contracting resources have become plenteous and cheap enough to fill that need. The company can move the business knowledge into more generic manager roles that are more useful to the company. (Roles with better pay, and job titles such as "software engineer" and "system analyst", and "project manager".) And the technical knowledge can be moved mostly out of the company into a "unpluggable" resource that they only need to pay as long as the work is actually being performed.
So, what's a programmer to do?
Well, first of all, stop being just a programmer. Don't be a code monkey. Yes, the under-appreciated coder-geeks of the world have "owned" that pejorative term. But in the eyes of corporate America, the words "code monkey" are always preceded by the words "just a". Code monkeys abound. They are replaceable. If you don't evolve (pun intended) you're going to end up obsolete when your company finds an offshore contractor that doesn't suck. (Yes, they do exist!)
One way you can evolve is by taking the advice of all the articles and reports I've linked to in this post. You can take courses, get certifications, etc., and become a "software engineer" or a "system analyst" or a "project manager". The numbers show there are plenty of companies out there that will be willing to pay you to do this as long as you don't suck at it. (And some that will even if you do suck. But I strongly advise against sucking.)
Many programmers go this route at some point. And there's no shame in it, if you can let go of coding every day (or at all) as part of your job and not hate yourself for it. I think I might go that route one day, but I'm not ready for it yet. For one thing I don't feel my experience base is either broad or deep enough to be as effective as I would want to be. But also, there are just too many cool languages and libraries out there. Too many programs that haven't been written yet. Too many problems that haven't been solved.
So what is there for me, and those like me? Those who don't want to give up programming as the core of our job, but don't want to end up teaching programming to high school kids by day while we sate our code-lust at night in thankless open source efforts? (No not all open source is thankless. But there are a lot of projects that go nowhere and just end up abandoned.)
Two answers:
- Consulting. Not everyone is willing to send the work around the world, dealing with language barriers, time-zone difficulties, and security uncertainties, to get it done. There's a definite place in the market for on-shore, face-to-face consulting. This spot is getting tighter though, so you had better make sure you're a top-shelf product.
- Software companies. The companies that sell the software that's putting code monkeys out of jobs are still raking it in. And they're actually solving new problems, building new technologies, etc. All the exciting stuff I dream of working on as I churn out yet another database script for a client.
A new kind of Democracy
Given the UW prof's area of focus, the conversation of course eventually came around, briefly, to blogs. She claimed that just as the Guttenberg printing press enabled the "democratization of knowledge", blogging will bring about another fundamental shift. One where, as before, some very nice things will be left behind, forgotten by most, in sacrifice to the emergence of a new mode of interaction that will bring about it's own useful new evolutions and societal impacts. I found this a tremendously interesting proposition, and hoped they would follow this line of thought a bit more, but unfortunately the host had latched onto some earlier point and swung back around to that, never to return.
I'm not sure I heard another word of the broadcast anyway though because my mind was off and racing.
Many people have already staked a claim on the blogosphere as the next frontier in journalism. Rejoicing has already begun that journalism will migrate into the blogosphere and morph into something new and different than ye olde journalisme, bringing enlightenment to all and ushering in the Age of Aquarius.... Certainly it will offer a new value proposition, but I am not yet convinced of the extent to which blog journalism will supplant traditional journalism. Regardless of that, however, these people are thinking too narrowly.
Let's go back for a minute to that statement about "democratized knowledge". I've heard this turn of phrase several times in reference to the rise of the printing press, but never really stopped to think about what is really being expressed there. The history of human communication seems to me to track very closely the history of significant "advancements" in civilization, and with a bit of thought I think it becomes abundantly clear why this should be.
When speech and body language were all that was available, the spread of knowledge was a social affair. You needed to have at least two people, in close proximity, interacting more or less directly. The advent of writing freed knowledge from the bounds of proximity in space and time. But consumption was serial because the recordings required a huge time investment. First one person, then another, then another.... If the recording was not lost or stored away in a private library. The printing press resolved these issues. It became trivial to make multiple copies of a manuscript. Copies could be made as long as there was demand for them. If some were destroyed others would survive.
The printing press made knowledge available to all those who desired to consume it. It was no longer the privilege of rich or powerful men. If you could read, you could learn, regardless whether someone was available to teach you personally. One man's idea could be made available to thousands, across time and space, and generally without concern for the social position of the consumer. This is what is meant by the democratization of knowledge, and it really was a significant turning point in human history.
I contend that the rise of blogging is the next great evolution of human communication. Sir Isaac Newton is quoted as having said "If I have seen farther, it is only by standing on the shoulders of giants." How do we know they were giants? Because they had the time and resources to make recordings of their ideas, and people valued them enough to demand and preserve them. In the past there were generally two ways you could get your ideas disseminated to humanity at large. If you were privileged enough to have the time and resources, you could finance a recording of your idea regardless of its perceived value. Or if you really believed in your idea, you could make a big sacrifice to get it recorded, and pray that people valued it enough to compensate you for your investment.
With blogging, these restrictions are largely abolished. If you have access to a computer, you can create a blog, for free, as I've done here. You can record your ideas as quickly as you can type them, and once you post it, it is instantly available for billions of people to simultaneously search for, stumble upon, consume, discuss, and share.
Newton and Einstein stood on the shoulders of giants. With blogging, we no longer have need of the giants. For you and I to see farther, we can assemble a great pyramid of normal human beings, each with a small contribution that in aggregate has potential that the "little people" of the past could only dream of participating in.
We can also move beyond ideas and into expression and experience. Blogging allows us all to share with the world our unique experiences, our viewpoints, our disasters, our epiphanies, our humanity. No matter how mundane our individualities may be judged by most, we now have the potential to find the other people out there who are weird in the same ways we are. With increasing frequency and ease we can now connect online with someone else who has shared our joys and sufferings, our beliefs and needs. Someone with whom we can identify and be comforted. Or we can peer into the "mundane" individualities of others, and experience the wide breadth of simple but beautiful diversities of humanity. These are often things that are difficult to get published in print, unless the presentation is particularly engaging, or the story is something that we expect will touch large groups of people. Online these barriers do not exist. And we need not simply consume the others' narratives either. We are empowered to interact with the speaker and participate in their narrative.
I could go on into more specifics, but instead maybe I will dedicate another post to that sometime down the line when I've thought about it a bit more.
To put it succinctly, Gutenberg's printing press enabled the democratization of the consumption of knowledge. With the emergence of blogging, we are witnessing the democratization of the production of knowledge and of expression of the human experience.
Maybe not the beginning of the Age of Aquarius.... But I think it's a bit of a "big deal" nonetheless. Though I am probably late to the party on this.
My Grand Entrance
I created this blog as an outlet, for my many and varied interests. It has been exceedingly rare to find someone who is willing to listen to me talk about them, let alone actually hold up one end of a discussion. So, hopefully this will provide me a pressure valve, where I can let out some of the thoughts that build up in my head without thoroughly irritating my friends and coworkers.
If I'm lucky, someone will read one of my posts all the way through before realizing they've wasted their time on the ramblings of some idiot on the internet.
And if I'm really lucky, it'll turn out that there's another person out there who actually cares enough about the same things as I do to tell me how ridiculous my ideas are and that I should just close up shop and go back to my day job.
Anyway, here are a few of the things I'll likely blog about:
- Software Development
- Programming Languages (and theory)
- Technology and Culture
- Theoretical Physics
- Education
- Math
- Professional Development
- Language / Linguistics