Tuesday, December 4, 2012

Essence of MVC

The Model-View-Controller pattern (MVC) was first described at Xerox PARC by Trygve Reenskaug in 1979:

MVC was conceived as a general solution to the problem of users controlling a large and complex data set. The hardest part was to hit upon good names for the different architectural components. Model-View-Editor was the first set. After long discussions, particularly with Adele Goldberg, we ended with the terms Model-View-Controller.

Trygve Reenskaug

Since then it has become the central pattern of virtually every application framework from the desktop to the web to mobile. I learned MVC from frameworks such as Rails and Cocoa, but my understanding lacked clarity. I wanted to reach beyond these artifacts and understand the essence of MVC, so I went in search of answers. After considerably more effort than I expected, I think I found some. This page is my current, best understanding of what MVC is and why it matters.

What is MVC?

Let's start with a simple, visual definition.

MVC is a pattern for taking a program...

and pulling it apart...

into modules with three well-defined roles:

The structure of a well organized MVC system is usually fairly simple. In general, there can be any number of model, view and controller modules. The communication channels between modules are restricted by an important rule: connections are only allowed between controllers and other modules. Non-controller modules should never talk directly to each other. Though not a rule, one more restriction is typically observed. Usually each view has only one controller associated with it.

NOTE: Advanced systems might use MVC hierarchically.

Now that we have a high-level idea of the structure of a MVC program, let's look at the three roles in detail.

The Model Role

A model's job is to hide and protect the user's important data. This is the data the user might want to access in 10 years - the data that has real-world significance. The model role draws a line between the data the user cares about preserving and the data that is temporary or can be re-generated from the models.

The internals of a model are responsible for storing the actual bytes that represent the user's data. Models are also responsible for ensuring the integrity and consistency of their data. Last, a model must provide efficient methods for querying and altering its data.

More than any other part of the system it is important to maintain a well defined API for each model. In order for a model to do its job properly, all methods of extracting and altering its data must be well defined, understood and respected.

The View Role

Views are the public face of a system. They determine how a user feels about the system. The attributes that make a view great are as much determined by psychology and aesthetics as engineering and science. A view's primary job is to communicate data to the user as effectively as possible. Their secondary job is to communicate the interactive nature of the interface.

Views encompass all aesthetic concerns of their system. All presentation assets, such as images, fonts, styles and layouts, belong in a view. Code that transforms data into pixels also belongs in the views. The view may even have hooks for specifying which elements are to receive user inputs, but all input processing should be done in the controller. The focus of the view is letting the UX designer worry about how the data and interface are presented.

The Controller Role

A controller's job is to process user inputs and coordinate the relevant views and models. In the simple case, it starts by extracting information from a models, doing some preliminary processing and then handing the processed data to a view to be presented. When new user inputs arrive, the controller validates them, performs alterations to the model on the user's behalf and updates the view.

The controller role's modular boundary is not as well defined as the view or model roles. Controllers do everything models and views don't. However, they tend to have a structure which is well defined by the system's use-cases. Controllers are organized around how the system will be used. They provide the glue between any number of views and any number of ways to extract and alter data from the models and align them to fulfill the user's needs.

MVC Roles and Relationships Diagram

This diagram summarizes the MVC pattern, each of the roles and their relationships:

Why use MVC?

With the roles of MVC defined, the next question is why do they matter. What are the benefits of MVC? How do we make use of MVC most effectively to realize those benefits?

There are three main benefits I have discovered.

Separation of Concerns

"The central idea behind MVC is code reusability and separation of concerns."

yiiframework.com

MVC identifies a very specific set of concerns that are repeated in virtually every programming project:

MVC concerns by function:

  • Model: safely and consistently store the data that matters to the user
  • View: present all, a subset or summarization of the data to the user
  • Controller: respond to user inputs and events, update the view and model appropriately

MVC concerns by expertise:

  • Model: data reliability, data consistency, query efficiency, update efficiency and storage-size efficiency
  • View: graphic design, information presentation and UX
  • Controller: expertise in the problem domain and coordinating the other concerns

MVC concerns by people:

  • Model: algorithm and data-engineers
  • View: designers and artists
  • Controller: experts in what the system needs to do

These are very different people. Sometimes we are forced to wear more than one of these hats, but ideally we would have different people responsible for each concern. MVC lets people with different expertise focus on what they do best.

Managing Complexity and Mental Modes

MVC is an abstraction, and like many abstractions it adds complexity. However, like good abstractions, it compartmentalizes complexity. MVC can dramatically reduce the complexity of each part of the system.

MVC is a special case of the general concept of modularity, and is a surprisingly good first-pass for designing a system's modularity. Why is it better than the alternatives? The core reason is the concerns MVC uses to define module boundaries are exceptionally well aligned with the real-world.

The MVC concerns closely parallel how we learn and think and the kind of specialists society produces. The model reflects a highly analytical mindset whereas the view requires the most creative thinking. The controller is the most political because it must balance the needs of many potentially conflicting concerns.

MVC is a good modularization pattern because it isolates three major modes of thinking. All the hard-analytical work of the system is isolated in models and can be done while ignoring everything else. The creative, intuitive and emotional work of a view can be done without worrying about any analytical concerns. The only sane way to make sure every user's needs get met is to ignore the details and instead focus on the big picture. That happens in controllers. Good MVC design not only modularizes code but it also compartmentalizes our mental modes for working on the code.

Decoupling & Evolution

Perhaps the most important aspect of MVC concerns the long term maintainability of code. MVC decouples the system's three main components allowing them to evolve independently.

In a monolithic system there are many opportunities for coupling artifacts. Changes to the visual layout, data-hiding or visual encoding are much more likely to effect data on disk. They may even break file backward compatibility. In the other direction, changes in how data is stored such as adding or removing fields, or restructuring for increased reliability, consistency or efficiency are in danger of breaking views or being exposed directly to the end user. Not only do changes anywhere in the system potentially effect all other parts of the system, but because of those large scale implications, in large projects any number of people may need to be consulted before making those changes.

MVC creates a strongly decoupled system. By minimizing the interface between each part and making sure each concern is handled by the right component, most changes only effect the component being changed. Decoupled views, for example, can change their presentation and even present new slices and summations of the data with no change to any models and little change to its controller.

Decoupling the model allows all kinds of changes to be made without touching controllers or views. It is nearly trivial to maintain backward compatibility when changing a file format: make two models, one for each version. It is much easier to add or remove information from a decoupled model. Placing all model related code inside a well defined module also makes it easy to make powerful changes to its performance. For example, it becomes much easier to add indexes for accelerating queries.

Sometimes it is impossible to make changes in a monolithic system that are trivial in a well modularized MVC system. Decoupling can make the impossible possible. Changes which were previously difficult can become easy, and programmer productivity typically skyrockets. The bottom line is model, view and controller modules can evolve much faster independently than they can as a monolithic whole.

An Alternative Description

Many great people have contributed to MVC over the past 30+ years. Apple has been a strong proponent of MVC design since 2002 with the introduction of OSX. The modern Macintosh operating system is based strongly on Objective-C which in turn is a marriage of C and SmallTalk. Since MVC was the "SmallTalk way", it's only natural for OSX to follow. Apple has a nice description of MVC:

The Model-View-Controller design pattern (MVC) is quite old. Variations of it have been around at least since the early days of Smalltalk. It is a high-level pattern in that it concerns itself with the global architecture of an application and classifies objects according to the general roles they play in an application. It is also a compound pattern in that it comprises several, more elemental patterns.

Object-oriented programs benefit in several ways by adapting the MVC design pattern for their designs. Many objects in these programs tend to be more reusable and their interfaces tend to be better defined. The programs overall are more adaptable to changing requirements - in other words, they are more easily extensible than programs that are not based on MVC.

Objective-C & Cocoa Documentation

The Essence of MVC

The question is not if you should modularize your system but how. Model-View-Controller is an excellent first pass. If you are programming in an object-oriented language you almost certainly should be using it.

The gospel of MVC as I've come to understand it comes down to three carefully chosen definitions with three key benefits:

Definitions of MVC

MVC is a modularization pattern that classifies all modules in the system into three roles:

  • Model: consistently store, query and alter the important data
  • View: communicate model data and user interface to human or computer clients
  • Controller: receive, interpret, validate, act on and respond to client inputs or other events

Benefits of MVC

  • MVC allows data-engineers, UX designers and system experts to focus on what they do best.
  • MVC is an excellent way to compartmentalize complexity and isolate mental modes.
  • The model, view and controller can evolve much faster independently than they could as a monolithic whole.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License. Copyright (c) Shane Brinkman-Davis, December 2012

Thursday, October 18, 2012

Literate Randomizer

Yesterday I was seeding my database for a new Rails project. This project has profile pages with large free-form text fields. In order to get a good sense of how the site layout feels with real data, I needed some decent fake profiles. The existing tools we were using just weren't cutting it.

There are several gems available to help with this task.

Here is a quick-link to Literate Randomizer's readme and source.

Faker

Faker is great for generating fake names and addresses, but when it comes to generating fake prose, it follows the Loren Impsum model of obvious nonsense. This was not going to cut it:

Mollitia id adipisci. A voluptatum et veritatis quas enim veniam consectetur. Id perferendis quia eum et. Rerum consequatur harum quisquam impedit sit et.

Qui est qui autem inventore esse laboriosam. Vitae expedita numquam eveniet repudiandae voluptatibus est in. Ratione eos maxime ut debitis illo. Voluptatibus ea molestiae et quas non est.

Et ut distinctio qui nihil consequatur odio. Labore voluptas magni sapiente totam. Sed sed est nobis quia temporibus pariatur. Nihil accusamus molestiae qui occaecati ut. Sed consequatur nobis quis consequatur rerum.

random_data

RandomData was next. RD did generate some more convincing prose, but if you glance the source you'll see it is simply randomly selecting from 23 fixed sentances

Soaring through all the galaxies in search of Earth flying in to the night. Every stop I make I make a new friend; Can't stay for long just turn around and I'm gone again. Hey there where ya goin, not exactly knowin'. Excepteur sint occaecat cupidatat non proident sunt in culpa qui officia deserunt mollit anim id est laborum. He just keeps on movin' and ladies keep improvin'.

Lorem ipsum dolor sit amet consectetur adipisicing elit sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. He's going everywhere, B.J. McKay and his best friend Bear. Ulysses, fighting evil and tyranny with all his power and with all of his might. Ulysses, fighting evil and tyranny with all his power and with all of his might. Every stop I make I make a new friend; Can't stay for long just turn around and I'm gone again.

Literate Randomizer

If you want to generate a lot of random prose, neither of the above options work well. With a little googling I ran across this post by Tim Riley. In it he shows some code for using Markov-Chains to generate plausible prose generated based on a source text. Using that source text, we can get a probability distribution of which words follow which words. Given that we can generate some near-english truly random sentences.

Note: RandomData has a Markov tool, too, but it is very limited. I found another gem that just copy-pasted Tim's code into a gem-file, but it wasn't properly packaged nor usable.

I was bummed to realize Tim hadn't made a gem of his wonderful little insight, so I thought I'd dive in and do it for him. I thought I could just copy Tim's code, give him credit, and release a nice clean gem. Turns out it wasn't that simple, or perhaps I just had too much fun with this problem. Another problem with all the tools above is they don't give you much control over the randomly generated output. I wanted control over the length and number of sentences as well as the number of paragraphs. I wanted interesting punctuation. With Markov chaining, you end up with a lot of sentences that end in prepositions (the, and, of, etc...).

The result, is a nice little gem called literate_randomizer. Here is a simple example. For more, please go to the literate randomizer source/readme.

 require 'literate_randomizer'  
 LiterateRandomizer.word  
 # => "frivolous"   
 LiterateRandomizer.sentence  
 # => "Muscular arms round opening of sorts while Lord John Roxton."   
 puts LiterateRandomizer.paragraphs  
The last line outputs:

Memory at the narrative may have already as I could be reflected upon. Society in a dozen. Headquarters for the stream. Stream which if we. Iguanodon's hide and admirably the right line of Project Gutenberg-tm electronic. Forgive me down hull-down on every detail which! Until they were. Mantelpiece in front of a necklace and that same scene. Eighteen of a creature beside his white merchant or hope.

Rain must be. Falls the paths which may have opened out we limped homewards sadly mauled and no. AGREE THAT THE CHAIRMAN. Twentieth century and horrible havoc. Train of Crayston Medal for canoes we are the scent and they. Swooped something which had a purpose than the envelope but if this. Entirely sir why this book. All these creatures that on through the rope dangled down the dinosaurs. Exhaust of the vertebrate stage of them and it began amid shouting gesticulating. Worthy of the old volcanic vents and distribute it away and under the prevailing. Assured that they took into line of the cliff. Unloosed a frugal breakfast of it run to the chasm which with. Loosened our thwarted wishes of dark until they took one's heart to win through. Cotton jacket which you can exhibit to gain a refund of hers outlined.

Novelist could have a jacket which communicated with its wing. Colliery explosion and stroking. Town was justified if ever moved upon his remarks had returned to bring. XIII A Sight which the plateau and many small Japanese tray of fairyland. Sadly mauled and flitted about that all turning these lumps of such. Cutting a sentiment. Grieved for us give. Whites our largest I should be confuted if animals then the prospect of the risk? Camberwell tram with a sheet of language is our zareba.

LiterateRandomizer.paragraphs options:
  • :first_word => nil - the first word of the paragraph
  • :words => range or int - number of words in sentence
  • :sentences => range or int - number of sentences/paragraph
  • :punctuation => nil - punction to end the paragraph with
    (nil == randomly selected from punctuation_distribution)
  • :paragraphs => range or int - number of paragraphs
  • :join => "\n\n" - join the paragraphs
    If :join => false, returns an array of the paragraphs

Thursday, September 27, 2012

Learnable Programming

More great stuff from Bret Victor: http://worrydream.com/LearnableProgramming/

Start by jumping through and watching the embedded videos. They are self explanatory. Watch them right now. It'll only take a few seconds each.

I started working along a similar line about a decade ago. I called it "interactive execution". The key idea is the program is re-run with every single change you make. All values are logged. You can see the actual inputs and outputs of every function for the entire run. You can see them update in realtime as you edit your code. It is easy to dismisses these ideas as non-applicable in "real-world-code":

These design principles were presented in the context of systems for learning, but they apply universally. An experienced programmer may not need to know what an "if" statement means, but she does need to understand the runtime behavior of her program, and she needs to understand it while she's programming.
A frequent question about the sort of techniques presented here is, "How does this scale to real-world programming?" This is a reasonable question, but it's somewhat like asking how the internal combustion engine will benefit horses. The question assumes the wrong kind of change.
Here is a more useful attitude: Programming has to work like this. Programmers must be able to read the vocabulary, follow the flow, and see the state. Programmers have to create by reacting and create by abstracting. Assume that these are requirements. Given these requirements, how do we redesign programming?

I believe it isn't that hard to imagine how to apply "interactive execution" to real-world code. Two simple tricks allow you to achieve useful application of these ideas in any sized (single-threaded) code-base:

  1. Allow the programmer to select a function as the start-point. Execute the program up to the start of that function and start a transaction against the entire running state of the code. Then, roll back the transaction with every change and restart execution from that start function.
  2. Run the execution in a separate thread from the editing thread. Every character typed aborts the execution and restarts it. If the execution takes a second or two, that's OK. When the programmer wants to see the current outputs, they just stop typing for a bit. If it takes minutes or hours, the programmer is in charge of picking a better start point for her transactional, interactive-execution session.

This is really powerful stuff. I've often noticed that the skill level of a programmer is proportional to how long they can go between coding and executing their code to see if it works. The key skill here is being able to simulate the computer in your head. What a waste! The computer can simulate itself much better than your head can. Let the computer do what it's good at, and leave the human mind open to being creative.

I liked this quote about OO:

Bob Barton [said] "The basic principle of recursive design is to make the parts have the same power as the whole." For the first time I thought of the whole as the entire computer, and wondered why anyone would want to divide it up into weaker things called data structures and procedures. Why not divide it up into little computers... Why not thousands of them, each simulating a useful structure?

I think I'll start calling objects "little computers".

Wednesday, September 19, 2012

How to Create a Turing Complete Programming Language in 40 Minutes

Using a programming language is cool. Writing a programming language is cooler. Writing one in 40 minutes rocks! I just released a screencast on how to create a Turing Complete programming language in 40 minutes and 70 lines of code using my Babel Bridge parser generation gem and Ruby.

In this example I use my Babel Bridge parser-generator gem: homepage / github

Here is the 70-line-source-code from the screencast.


We continue to need fresh minds and new approaches to programming. There are many great challenges facing us in improving programmer productivity and supporting programming in our massively parallel age. I hope this screencast wets your appetite to pick up a parser-generator and play around with your own language. Maybe you'll invent the next Ruby, Java or Javascript!

Monday, March 19, 2012

Bret Victor - Inventing on Principle


I want to recognize excellence. Bret Victor's talk on Inventing on Principle is excellent. Go watch it now if you haven't already. vimeo-link

I loved how the first half exactly paralleled thoughts I've had for a long time which I've summed up as POET: the Principle of Exposure & Transparency - any problem becomes trivial with enough exposure. I was thinking more from the solving-problems angle where Bret Victor was thinking from the creativity angle. I think the ideas mirrored nicely. I definitely liked how he visualized interactive-execution in text-code. That's a nut I hadn't cracked yet. This is just the tip of the ice-berg. Lots of interesting work to be done here.

The principled approach of the second half of Bret's talk was fascinating. I've had a focused path for most of my career, but even still, after watching his talk I've had a hard time simplifying my path down to a single principle. I'm not sure if that's because I don't fully understand my path's essence as well as Bret understands his or if some goals are not distillable down to a one-sentence description. I like his point regarding how a good principle can give you guidance where-as a bad one can be meaningless. "Write beautiful code" - meaningless. "Creators need immediate connection with what they are creating" - meaningful and provides a system of measuring, giving feedback and guiding you through design-space.

I feel pretty strongly about eliminating "artifacts". Historical legacies that are no longer relevant can gum up the works, particularly when they are hidden in plane sight. I also love to invent. As a contrast to eliminating artifacts, I am also drawn to "essential" technologies. What technologies, if they existed, and were done "right" (with as little artifacts as possible) could change the world? I suppose in that light, my Essence & Artifact theme might just be my core principle. Eliminate artifacts; understand the Essence. One might argue it's closer to the meaningless end of the principle continuum. It certainly isn't as concrete as "No Modes". On the other hand, every feature, function, design and architecture can be measured against an Occam's Razor of "is this Essential, or is it Artificial?"

Essence is timeless, simple, powerful, maximally applicable, modular, and focused. But be careful, the purely essential is also purely theoretical.

Artifacts are historical, complex, constrained, difficult, narrowly focused and/or un-focused. But be respectful, artifacts are the only things that are actual.