October 2007

Asynchronous messages as OOP

Originally I wanted to write about cargo cult programming, but I read Michael Feathers latest post and the idea was too good to let it pass by.

Since I'm currently struggling with creating a component system, which should serve as the base for a somewhat large domain model, I'm playing with thoughts about being able to see each component as its own small server, which can be located on some arbitrary system. This leads to a distributed system (like Amoeba as operating system), with all its advantages and drawbacks. My biggest problem so far is the communication between the components:

• either I have RPC styled services, which results in a SOA-styled architecture (which would be really awful
• or I have real messaging between objects, and must deal with all the problem which can result from a missing answer to a message

Michael solves this with removing the need for answers to messages - messages just generate more messages, and somewhere in the end the right things happens. Surely this is the essence of OOP - Alan Kay not only said the OOP for him is just passing messages between objects, he also said that on the Internet (as the prime model for a distributed system) each object should have its own IP (or at least its own URL)

But since my domain model system needs to react to requests from users (it should serve as the base for large web applications), for now I need some kind of answers to my messages. But for that, I can always use blocking and wait for the messages to flow back to the original requester

I will need to think more about that - the advantages of this model are way too large to miss them, and I hopefully can deal with all the consequences.

Handling distributed teams

Both Turmalix and Mark Levison discuss the problems faced by locally distributed teams. For a current project one half of the team is located in Melbourne, Australia, the other half works in Germany. Out biggest problem - the information flow. If a team sits in a common location, you have many ways of getting information: just go to someone and ask him directly, have regular short meetings, maybe you overhear a nearby conversation, or you meet someone while getting a coffee. All of that is missing now, and the german team sits, literally spoken, in the dark.

The time difference makes things even worse. Currently Melbourne is always 8 hours ahead, after DST this will be 10 hours. We have a windows of several hours only because one team works constantly overtime, and the other people try to start early enough in the morning to allow at least for a daily information sync.

But if there are any unforeseen problems, and somebody needs help from the other team, he normally needs to wait until the next day. This especially requires care when you are checking code into the repository - we had too many cases of non-compiling code, and this can disrupt the work of a whole team for a day.

Our most important tool so far is ICQ, because it allows instant communication like the telephone, but can be ignored if you are busy or something else needs your attention. We try to have regular phone calls (because speaking is faster than typing :), but the heavy schedule doesn't always allow for that.

Fewer people are faster

Smoothspan writes: To Build Better Software, You Need Fewer People. He talks about the problem that larger teams tend to be slower because of all the communication issues between them. 10 Developers seem to be a reasonable limit for a software team, but I think there are tasks where the limit is even lower. I'm currently working on a team trying to develop a new architecture for a whole software system. We started with about 8 engineers, and it was impossible to even reach a conclusion about the requirements for the new architecture. When the team shrunk to 4 people. with 2 sharing the main work, it was a matter of a few days to get the ideas flowing. We still have the larger team in the backyard, and we will need to defend our result in presentation, but the main work is not restricted to all the goals these people are trying to reach. With 2 people sharing their ideas on a daily basis (while working on different layers of the architecture) we were able to define not only the goals in a nice way, but also to define the architecture in a comprehensive and understandable way - something which a committee will never be able to to (think about all the WS-Deathstar mess). It helps that we are standing on the shoulders of giants, and reading papers from Richard P. Gabriel and Alan Kay was a huge inspiration.

But back to the team size - the limiting factors are communication and the different goals. There is no simple way to have all the ideas of 8 or more people flowing smoothly between all of them, discussing and criticizing them. This is even problematic with 4 people, like in our team. The other limiting factor could be solved by a (maybe benevolent) dictator who just decides what the goals of the project are. But if you want to reach a coherent set of goals in a committee you will need a really long time.

So, for development 10 people may be the upper bound for a working team, but for doing creative work like designing a system architecture, I think 3 is the best size. One person can run into all the wrong directions, and 2 person could be stuck because they cannot resolve their differences. But 3 people can communicate effectively, and should be able to resolve all problems in a short time.

FORTH programs are also DSLs

I explained in the last entry how I see Lisp programs as domain specific languages (or rather, how Lisp as a language encourages the creation of all programs as DSLs). When reading the explanation of Richard M. Jones for his minimal FORTH compiler, I was reminded that FORTH plays in the same league. You write your program by building new words basing on the already existing words, and try to capture the problem domain with them. In the end, you should have a minimal set of words describing your program. This is just another description of a domain specific language.

When talking about programming languages, I have no real idea how to look at Smalltalk. On the one it uses a very small set of keywords (5) and concepts (namely message passing and assignment), and builds everything from there. But I cannot see how it encourages the wrinting of programs as more specific problem descriptions the way Lisp and FORTH do it.

Thinking about model driven software development

I hate buzzwords, and MDSD is one of them. No so much because as an bad idea, but because its so often misunderstood. When someone talks about model-driven development, he usually means two things:

• drawing some pictures (or diagrams, if a tools is used)
• he can solve all development problems with these pictures, and there is no need for further abstractions

I came to think the see these pictures as a special kind of domain specific languages. Its graphical, and will therefore require special tool support (as long as don't want to create code, you can draw them on a flip chart but generating code is what MDSD is all about). This also means that its always difficult to create these diagrams from another tool (meaning that you have a higher language).

The last thing is the main point - DSL are created to solve some problems easier than in a general programming language. The DSL abstracts some things away, and make other things easier on its way. But this doesn't mean that it is the end of the road - for some even more specific problems it can be a good idea to create an even higher abstracted DSL, which generates code in the former DSL. If doing so is easy, this encourages the creation by the developers. This can be seen in the Ruby & Rails community - as Jamis Buck stated: most well-written Ruby programs are already a DSL. For me thats one reason development in Ruby is so fast, and so much fun.

Of course, Ruby did not invent this. Lisp with its defmacro did this some decades earlier, and took it to a much more extreme level: since data is code, and code is data, every DSL specified in Lisp is also Lisp code - which means that you need no special facilities for code generation, and you get all the power of Lisp on all levels of abstractions you want to create. To change Jamis' quote: in Lisp its not possible to write a program which is not a domain specific language.

For my journey with Common Lisp, this means that I now need to write Lisp programs :)

Learning LISP

I wanted to learn Lisp for some years now, but I never got around doing it. Maybe its just that Lisp is too different from all languages I have used so far (which includes COBOL, Forth, but was mainly Pascal, C++ and Java for the last 8 years). Maybe its the bad memories to my time in university - we had a course on artificial intelligence there. We spend the 3 months assigned to learning Lisp just with fiddling aroung with lists and CAR and CDR (the other 3 months were more useful - we used PROLOG to model relationships (mother, father, child etc.)).

But I knew that at some point I would need to learn it - there are just too many good concepts in there , and I would need to learn them. So, when I stumbled across Practical Common Lisp, it was the right time. I was already learning Scala (which is the nearest thing for any Java developer trying to get his head around functional programming), so the step to Lisp seemed not so big this time.

Installing LispBox was a small adventure - on Windows it wants to create some directories on drive C, which is not allowed if you have no administrative rights. On Ubuntu you can download everything via Aptitude, but need to activate SLIME manually in Emacs (which is explained nicely in the manual). After I figured out everything it works like a charm.

So I'm currently working through chapter 5, "functions", and learned already my first lesson: code generation, when done right™, can be a really powerful tool for better code.