Concerns about Principles and Change in Software Design

Computer pioneer and genius
Edsger Wybe Dijkstra

High quality software designs are easier and safer to work with. Investing in quality demands discipline and skill. Good design principles are usually helpful. But, are they always applicable?

This post investigates the applicability of the “Open-Closed Principle” when we add new functionality to a software design whose source code is entirely under our control.

Our argument is based on Dijkstra’s concept of separation of concerns.

The Open/Closed Principle (OCP)

The “Open/Closed Principle” (OCP) was first advocated by Bertrand Meyer in the first edition (1988) of his book “Object Oriented Software Construction” and is considered by Bob Martin as “the most important principle of object-oriented design”.

OCP states that modules should be open for extension and closed for modification. New functionality should be implemented by adding new classes, attributes and methods, instead of changing the current ones.

The simplest way to apply OCP is to implement the new functionality on new derived (sub)classes that inherit the original class implementation. Another way is to mediate client access to the original class with an abstract interface, so new functionality can be implemented on new classes that are accessed through the same interface. Both ways create new classes and leave the original implementation untouched.

Modular and Expressive Abstractions

High quality software designs are made of modular and expressive abstractions.

Modularity is a design quality a syntactical level. A design’s abstraction is a module if its internal implementation and external context are coupled only through the specification of its responsibilities.

Modular designs enable its abstractions to be locally implemented and globally reusable. Modularity facilitates human minds to engineer highly complex and adaptable systems.

Expressiveness is a design quality at semantic level. A design’s abstraction is expressive if its specification represents a domain concept.

Expressive designs can easily articulate abstractions to clearly express the relevant concepts, problems and solutions. Expressiveness facilitates understanding and communication that enable human minds to engineer relevant and valid solutions.

Identifying, representing and separating relevant concerns is the fundamental principle for our design process.

Concerns bridge syntax and semantics to define abstractions and assign responsibilities that compose and decompose modular and expressive designs.

When OCP should be Deliberately Broken

When our domain model, system requirements or design objectives change radically, new concerns unfold and emerge as relevant.

The new concepts, problems and solutions can only be expressed after the respective concerns are also represented and separated in the design, otherwise we would introduce duplication and break modularity.

The emerging concerns must be distilled from all the modules where they were mixed with other concerns. That is when we deliberately break the OCP, opening and changing previously existing classes in order to represent and separate the new concerns.

New relevant concerns can emerge from either small or big changes. Fowler’s Video Store didactic program provides us a good example. Creating an HTML version of the customer’s statement makes its format a relevant concern. The statement method must be opened and factored to avoid duplication. After format is separated from content, the OCP again must be observed when creating new versions for other formats.

Special thanks to Kent Beck for the enlightening inspiration!

Picture from WikiMedia Commons.

Revisiting Fowler’s Video Store: Refactoring Code, Refining Abstractions

Refactoring techniques and Design Patterns have been extensively propagated and advocated for over a decade now. Why we still find hard to change, anaemic or overly complex object oriented structures? What is missing in our design practices?

In this series, we use Fowler’s didactic Video Store program to show practical advanced refactoring strategies that improve design simplicity, expressiveness and flexibility beyond standard object-oriented practice.

Refactoring and Design Patterns

Refactoring is a modularity improving technique that is essential for any evolutionary software development process and indispensable in the skill set of any professional software developer.

In the extremely influential book “Refactoring: Improving the Design of Existing Code”, Martin Fowler defines Refactoring as “a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behaviour”.

Fowler’s Refactoring book and the GoF Design Patterns book have played a fundamental transformational role on the professional community, educating a generation of software developers on their move from the procedural to the object oriented design paradigm. More recently, Joshua Kerievsky published Refactoring to Patterns, leading refactoring to a new level of abstraction by explicitly combining both approaches.

Despite all these resources, why we still see complex object oriented structures, contrary to the objectives of simplicity, expressiveness and flexibility, also implied by the refactoring definition?

What is Missing?

High quality software designs are made of modular and expressive abstractions.

Refactoring and design patterns are misused by a large number of software developers. There is an excessive focus at the syntactical level. A broader and deeper perspective is needed when analysing modularity problems and directing the refactoring process towards greater relevance to its design objectives.

The missing points are:

1. each refactoring step is an opportunity for refining abstractions;
2. abstraction refinements should improve both modularity and expressiveness.
   

In summary: refactoring processes should consider the program’s context and domain semantics to explicitly rethink both its design structure and abstractions at each refactoring step.

Practical Strategies for Effective Design Improvement

Effective refactoring transcends the program’s text and syntax to improve modularity and keenly enhance simplicity, expressiveness and flexibility.

Fowler’s book introduces refactoring techniques with a small program for a video store, whose main function is to print a statement of a customer’s charges and his frequent renter points.

In this series, we use Fowler’s very simple and rich didactic example to illustrate the following refactoring strategies:

1. Adopt Object-Oriented Modules. Apply proper object-oriented design to reduce syntactical noise and make programs easier and safer to work with. Prepare your program to implement most new requirements by small and focused interventions, without generating unintended side effects.
2. Reveal the Relevant Domain Concepts. Listen carefully the domain experts to reduce semantic noise by making their relevant concepts explicit and evident in all design expressions.
3. Align with Contextual Variance. Confront program invariants with contextual variances to help make design decisions that improve consistency and flexibility.
4. Build Resonant Abstractions.

These refactoring strategies have been consistently beneficial to our software design practice over the years. In case you haven’t yet, we strongly suggest you try them in your daily software development activities. We will be very happy to listen to your feedback with ideas and experience about software design and refactoring.

Revisiting Fowler’s Video Store: Variants and Invariants

Glove: supple and protective

“Nothing endures but change.”

Heraclitus (540-480 BC)

“A well-worn glove becomes supple at the points where the fingers bend, while other parts are stiff and protective.”

Domain-Driven Design, by Eric Evans

In our design and refactoring efforts, we can’t afford to join the epistemological quest for Heraclitus’ supreme Logos – the deep rational principle that rules all changes in the universe.

We can and should, however, reason about how change affects consistency in our software designs. For stateful programs, such as object-oriented software, this necessarily brings us to understand its variants and invariants and how they are translated to design flexibility and coherence in the face of contextual variance.

Our objective is to reflect on how to design software that fits our domain like a well-worn glove. Simple, flexible and durable. Easy to work with. Capable of bending without breaking where we most need. Consistent and adaptive in the face of change.

Program’s Invariants in the Face of Contextual Variance

An invariant is a logical expression that remains true throughout a sequence of transformations. Invariant analysis is traditionally employed for software design on a textual basis. Invariants are defined or identified and then verified for representation structures, iterations, objects and classes.  Here, we want to go beyond what is present in the software texts (the programs themselves).

This post resumes the Revisiting Fowler’s Video Store series. After making the relevant domain concepts explicit, we focus on another aspect of domain semantics, studying contextual variance under the perspective of the following issues:

1. The Passage of Time;
2. Changes in the Video Classification;
3. Changes in Rental Prices and Terms.

The Passage of Time

The first variance in the context of any software system is the inexorable passage of time. In our case, it affects directly the video store rental periods, which are represented by a record of the number of days a video has been rented. This representation is not persistent: it needs to be incremented at the beginning of each business day, otherwise it would become invalid. Therefore, the representation is not consistent because it is incoherent with the program’s context.

We propose to replace the number of days by the initial rental date and to calculate the rental period through date arithmetic. The refactoring is internal to the Rental class, so there is no need to change its specification (or published interface).

The consistent rental period calculation can be readily implemented using the new RentalCalendar class, that applies the time point pattern with day granularity specifically for video rental business calendar. The alternative Rental class constructors and getDaysRented method are shown in the figure below.

public Rental(Video video, RentalCalendar initialDate) {

    _video = video;
    _initialDate = initialDate;
}

public Rental(Video video, int daysRented) {

    _video = video;
    _initialDate = RentalCalendar.pastDay(daysRented);
}

public int getDaysRented() {

    return _initialDate.daysTo(RentalCalendar.today());
}

Two additional reflections on this refactoring:

1. the initial rental date is another relevant domain concept that we made explicit in our program;
2. it confirms again an obvious truth: we should only persist what is persistent.

Changes in the Video Classification

Both the initial Video Store dicdatic program and its purely syntactical refactoring use a static classification scheme for pricing information. In the initial program, each video is  associated with a price code; purely syntactical refactoring replaces this association with an association to a price subclass.

There is an apparent contradiction in this model that should be questioned and clarified:

“Do New Releases stay in this category for ever?”

If the domain expert answers “No. Of course not.”, then the next question should be:

“If a video is reclassified, how does it affect the calculations for its current rentals?”

Programs must be perfectly adherent to all relevant business rules. If the domain experts affirm the calculations should remain the same as when the customer rented the video, we should transfer the pricing information from the  Video class to the Rental class, because it is transient to Video and persistent to Rental.

In the previous post, we introduced the Conditions class, created to represent the domain concepts that regulate the price and frequent renter points. Now we just need to adapt the Rental class constructors and its getCharge method, as shown in the following program fragment.

public Rental(Video video, RentalCalendar initialDate, Conditions conditions) {

    _video       = video;
    _initialDate = initialDate;
    _conditions  = conditions;
}

public Rental(Video video, int daysRented, Conditions conditions) {

    _video       = video;
    _initialDate = RentalCalendar.pastDay(daysRented);
    _conditions  = conditions;
}

public double getCharge() {

    return _conditions.getCharge(this.getDaysRented());
}

Changes in Rental Prices and Terms

I enjoy watching a video every week at home. I usually rent two or three for my family at my local video store. This week I saw something that made me rent seven videos! Look yourself at the picture below.

Simultaneous Rentals Promotion

I think the store owner finally understood my needs and the result is more business and satisfaction. For us, here, this is a good example of change in rental prices and terms. For regular and children’s videos, the rental conditions now depend on the number of simultaneous rentals. Each additional video rent extends the rental term by one day. So, the initial term is 3 days for 3 videos, 4 days for 4 videos and so on up to 7 videos.

In the original didactic program the prices and terms of rental for each type of video are expressed as literals embedded in the calculation formulas. Standard object-oriented design practice refactors the conditional expressions into a specialisation solution based on inheritance and polymorphism that implements the strategy design pattern through a class hierarchy.

We replaced the conditional expressions by parameterised unified expressions, where the parameters are domain relevant concepts, originally implicit and then explicitly articulated as rental conditions. The program fragment below shows how easily the rental conditions value object can be moulded to face the changes in rental terms.

public Collection<Rental>
simultaneousPromotion (Collection<Video> videoCollection, RentalCalendar initialDate) {

    int simultaneous = videoCollection.size();
    List<Rental> rentalCollection = new LinkedList<Rental>();

    if (simultaneous >= 3 && simultaneous <= 7) {

        int initialTerm = simultaneous;
        Conditions conditions = new Conditions(initialTerm, 3.00, 1.50, 1);

        for (Video eachVideo : videoCollection) {

            Rental rental = new Rental(eachVideo, initialDate, conditions);
            rentalCollection.add(rental);
        }
    }

    return rentalCollection;
}

As we can see, there was no need to change the calculation logic. The refactoring solution based on making relevant concepts explicit is not only simpler and more expressive, but also and more flexible and consistent in the face of contextual variance.

Conclusions

Confronting the program invariants with relevant contextual variances illuminates the refactoring process. As usual, we should always listen the domain experts, strive for simplicity and use a maintenance need as an opportunity for refactoring.

In this post, we brought contextual variance to our refactoring process. Working through the program invariants in the face of contextual variance generated two contributions for our refactoring of the Video Store program:

1. it made apparent the need to refactor the representation for the rental periods;
2. it confirmed the value object based refactoring of the program’s conditional expressions.

Refactoring for coherence with relevant contextual variances improves the software design in terms of consistency and flexibility. Just like an well worn glove, our refactored program is now more consistent and flexible exactly where it makes most sense.

Design coherence at the syntactical level is achieved by colocating data and functions responsibilities. At semantic level, coherence means consistency and flexibility in the face of domain dynamics. This requires deeper thinking about the relevant contextual variance and the relevant domain concepts.

Revisiting Fowler’s Video Store: Making Implicit Concepts Explicit

Not just a Gear, a Clock!

“A deep model contains the central concepts and abstractions that can succinctly and flexibly express essential knowledge of the users’ activities, their problems and their solutions.”

“Ultimately … this should make the software more in tune with the way the domain experts think and more responsive to the users’ needs”

Domain-Driven Design, by Eric Evans

Domain-Driven Design situates the domain model in a central and convergent role, tightly articulating semantics and syntax for problem definition, solution design and software implementation.

In this post, we bring the domain semantics to our refactoring process. We want a deeper perspective to analyse modularity problems and to direct improvements towards greater relevance to our design objectives. So, let’s resume the Fowler’s Video Store refactoring and explore alternative design decisions based on greater emphasis on model semantics.

Fowler’s Video Store didactic program has two main functions: rental calculations and customer statement printing. This post focuses on the domain concepts that are most relevant to the rental calculations. Next posts will explore additional design alternatives for rental calculations and customer statement printing.

Initial Rental Calculations Design

Rental calculations include charge and frequent renter points. Both calculations are a function of the rental period and whether the video is either a new release, for children or regular.

The initial program defines a price code attribute for each video; this information is used to select the right calculation formulas embedded in switch programming statements inside the customer statement method. The complexity of such design grows as new price codes are added. It becomes specially dangerous if the statement method is duplicated to implement the HTML format.

Refactoring with Inheritance and Polymorphism

As we saw previously, the refactoring book evolves this design solution first by extracting the calculations to the methods getCharge and getPoints and moving them to the Video class.  This eliminates the need of duplicating the formulas when creating a new statement method.

The next step is to isolate the formulas from the Video class, replacing the price code with a new Price class. This class is then further specialised into the NewRelease, Children or Regular subclasses, so that the price code can be replaced by a polymorphic Price reference to an object of the respective subclass.

The following figure shows the design solution adopted by the refactoring book.

Video Store - Price Specialisation

The recommended refactoring improves modularity by using inheritance and polymorphism to adopt a typical object oriented syntax, that implements the strategy design pattern. The refactoring steps end up constituting the Replace Conditional Logic with Strategy refactoring pattern.

It looks like the right way to go. Doesn’t it? Well, let’s explore a design alternative.

Finding Relevant Implicit Concepts

The rental model for prices and terms is implicitly codified in the following formulas scattered along the Price class hierarchy implementation:

regularCharge = 2.0;
if (daysRented > 2) regularPrice += (daysRented - 2) * 1.5;

childrensCharge = 1.5;
if (daysRented > 3) childrensCharge += (daysRented - 3) * 1.5;

newReleaseCharge = daysRented * 3;

If we look closely enough at these formulas, we discover they are actually three instances of the same formula:

charge = initialPrice;
if (daysRented > initialTerm) charge += (daysRented - initialTerm) * extraPricePerDay;

The implicit concepts of initial term, initial price and extra price per day have just emerged as relevant to our design! They define a clearer, explicit, flexible and unified pricing model, replacing three static functions by a single meaningfully parameterised function. The same is true for the frequent renter points calculations. The emerged concepts can be articulated as rental conditions.

Naturally, all relevant domain abstractions must be institutionalised as part of the language effectively practised in the business and expressed in the software. Therefore, emerging concepts must be presented, discussed, reviewed where necessary and validated with the business domain experts.

Making the Relevant Concepts Explicit in the Design

As part of this this convergence process, we propose to explicitly define the Conditions value object as a substitute for the Price class hierarchy. The parameters initial term, initial price, extra price and extra points are invariant for regular, children’s and new releases. Therefore they can be defined at construction time:

Conditions regular    = new Conditions(2, 2.0, 1.5, 0);
Conditions childrens  = new Conditions(3, 1.5, 1.5, 0);
Conditions newRelease = new Conditions(1, 3.0, 3.0, 1);

The parameter daysRented is defined at calling time for the functions getCharge and getPoints.

public class Conditions {

  private final int    _initialTerm;
  private final double _initialPrice;
  private final double _extraPricePerDay;

  private final int    _INITIAL_POINTS = 1;
  private final int    _extraPoints;

  public Conditions (int initialTerm, double initialPrice, double extraPricePerDay, int extraPoints) {

    _initialTerm      = initialTerm;
    _initialPrice     = initialPrice;
    _extraPricePerDay = extraPricePerDay;
    _extraPoints      = extraPoints;
  }

  public double getCharge (int daysRented) {

    double charge = initialPrice;
    if (daysRented > initialTerm) charge += (daysRented - initialTerm) * extraPricePerDay;
    return (charge);
  }

  public int getPoints (int daysRented) {

    int points = _INITIAL_POINTS;
    if (daysRented > initialTerm) points += _extraPoints;
    return (points);
  }
}

After all changes the new domain model implementation is:

Video Store - Rental Conditions

Conclusions

Refactoring at the semantic level improves the matching between business, model and implementation, adds flexibility where needed and addresses the major stakeholders’ concerns. As usual, we should always listen the domain experts, strive for simplicity and use a maintenance need as an opportunity for refactoring.

The initial program calculates rental charges and frequent renter points as conditional expressions. Standard object oriented design practice focuses on the conditions to specialise the expressions and implement the strategy design pattern for each calculation. This post focuses instead on the expressions themselves to identify the relevant domain concepts that generalise the expressions.

Bringing domain semantics to the process enabled us to propose a Extract Value Object refactoring, as an alternative to the Replace Conditional Logic with Strategy refactoring, with the following benefits:

• decoupling: duplication in the multiple formulas is eliminated;
• expressiveness: changes can be more clearly and easily applied;
• simplicity: the program’s design structure is simplified.

Programs that instantiate an Anaemic Domain Model can be greatly improved by simple and straightforward refactoring aimed at adopting proper object oriented syntax. Additional improvements, however, require deeper thinking and carefully considering the domain model semantics in order to select the course of action from multiple design options.

Picture by Readerwalker’s photo stream. Original at Flickr.

Revisiting Fowler’s Video Store: Adopting Object Oriented Syntax

Have you ever got a program that you needed to add, change or even fix a simple function, but you just didn’t know where to start or were afraid of breaking the program’s logical consistency?

Refactoring is a modularity improving technique that is indispensable in the skill set of any professional software developer. In this post, we use Refactoring to effectively adopt an object-oriented design syntax and make a program easier and safer to work with.

The extremely influential book “Refactoring: Improving the Design of Existing Code” introduces the technique with a small didactic program for a video store, whose main function is to print a statement of a customer’s charges and his frequent renter points.

The Initial Program

The initial program was structured with classes representing the domain concepts of Customer, Rental and Video. The statement method in the Customer class is reproduced below (updated to current Java syntax).

    public String statement() {

        double      totalAmount  = 0;
        int         renterPoints = 0;
        String      result       = "Rental Record for " + getName() + "\n";

        for (Rental each : _rentals) {

            double thisAmount = 0;

            // determine amounts for each line
            switch (each.getVideo().getPriceCode()) {
                case Video.REGULAR:
                    thisAmount += 2;
                    if (each.getDaysRented() > 2) {
                        thisAmount += (each.getDaysRented() - 2) * 1.5;
                    }
                    break;
                case Video.NEW_RELEASE:
                    thisAmount += each.getDaysRented() * 3;
                    break;
                case Video.CHILDRENS:
                    thisAmount += 1.5;
                    if (each.getDaysRented() > 3) {
                        thisAmount += (each.getDaysRented() - 3) * 1.5;
                    }
                    break;
            }

            // add frequent renter points
            renterPoints++;

            // add bonus for a two day new release rental
            if ((each.getVideo().getPriceCode() == Video.NEW_RELEASE) &&
                (each.getDaysRented() > 1)) {
                    renterPoints++;
            }

            // show figures for this rental
            result += "t" + each.getVideo().getTitle() +
                      "t" + String.valueOf(thisAmount) + "\n";
            totalAmount += thisAmount;
        }

        // add footer lines
        result += "Amount owed is " + String.valueOf(totalAmount) + "\n";
        result += "You earned " + String.valueOf(renterPoints) +
                  " frequent renter points";

        return result;
    }

According to authors Martin Fowler and Kent Beck, a long method is a “bad smell in code” indicating “there is trouble that can be solved by a refactoring“. A quick analysis reveals the method is directly responsible for all of the following functions:

• calculating the charge and frequent renter’s points for each customer’s rentals;
• calculating the total charge and frequent renter’s points for the customer;
• generating and formatting the statement.

A throughout program analysis confirms there is a modularity problem: it is not really an object oriented design. The classes are just data structures and the program’s main functionality is lumped into one single method. Take a look at the class diagram.

Video Store - Initial Class Diagram

The dissociation between data and function responsibilities in a object oriented design is a modularity problem at the syntactic level. It means there is a problem in how the modules are composed and combined to form the program. This type of problem is usually indicated by the following symptoms:

• long methods;
• one object manipulating data defined in other objects;
• long object/method chains intermediating access to other objects.

Modularity problems at syntactic level typically emerge as high priority issues as soon as any maintenance need touches them. Fortunately, they usually can be detected from program code analysis and solved simply by applying basic object-oriented design rules.

New Requirements Expose the Design Problem and Ignite Refactoring

Using Fowler’s own words, “there’s nothing wrong with a quick and dirty simple program”, as long as there is no need to change it. As expected, the program’s bad design became a relevant problem when the users asked to create an HTML version of the customer’s statement and told other changes would follow in the rules for video classification and rental calculations.

Although the change in the customer’s statement is confined to format and does not affect content, it is not possible to reuse the current statement generation and just add a specific method for the new HTML format.

There are two ways to immediately add the new function:

1. duplicate the conditional logic for the intended format at each output point inside the statement method;
2. duplicate the statement method itself, including the content generation logic for calculating charges and frequent renter points.

Duplication is unacceptable, because it adds complexity and threats the next changes. Replication breaks the program modularity by introducing coupled parts to the program’s structure. Whenever one of these parts changes, the respective couples must also change simultaneously. The resulting program would be harder to change and prone to inconsistency.

We should, instead, follow our masters advice:

When you find you have to add a feature to a program, and the program’s code is not structured in a convenient way to add the feature, first refactor the program to make it easy to add the feature, then add the feature.

Before jumping into the program’s code and making any changes, necessary precautions must be followed by any professional software developer:

1. identify precisely the scope of the intervention: what is to be changed, what is to remain invariant;
2. prepare, review and put in place solid tests covering both changes and invariants.

Cohering Data and Function Abstractions

The most basic design rule for object-oriented design is to use the classes as the loci for cohesive abstractions integrating data and function responsibilities. The statement method is a clear target for this rule.

Fowler decomposes the statement method into elemental functions relative to the classes already defined in the program and then delegates each of these functions to the appropriate class. The class responsibilities rearrangement is evident comparing the class diagrams before and after the refactoring.

Video Store - Rearranged Responsibilities after Refactoring

The transformation is very beneficial to the statement method, now renamed textStatement. It became easily understandable after delegating all the logic for calculating charges and frequent renter points and restricting its responsibility to content generation and formatting.

    public String textStatement() {

        String result  = "Rental Record for " + this.getName() + "\n";

        for (Rental each : _rentals) {

            // show figures for this rental
            result += "t" + each.getVideo().getTitle() +
                      "t" + String.valueOf(each.getCharge()) + "\n";
        }

        // add footer lines
        result += "Amount owed is " + String.valueOf(this.getCharge()) + "\n";
        result += "You earned " + String.valueOf(this.getPoints()) +
                  " frequent renter points";

        return result;
    }

Refactoring the program made it easier and safer for us to add the new feature by creating the new htmlStatement method for the HTML format. The program still exhibits some duplication, but complexity was managed to a more reasonable level.

    public String htmlStatement() {

        String result  = "<P><H1>Rental Record for <EM>" + this.getName() + "</EM></H1></P>\n";

        while (Rental each : _rentals) {

            // show figures for this rental
            result += "t" + each.getVideo().getTitle() +
                      "t" + String.valueOf(each.getCharge()) + "<BR>\n";
        }

        // add footer lines
        result += "<P>Amount owed is <EM>" + String.valueOf(this.getCharge()) + "</EM></P>\n";
        result += "<P>You earned <EM>" + String.valueOf(this.getPoints()) +
                  "</EM> frequent renter points</P>\n";

        result += "</BODY></HTML>";
        return result;
    }

Conclusions

The initial Video Store didactic program instantiates the infamous Anaemic Domain Model, the archetype of a procedural design disguised under a object oriented appearance. Anaemic models gradually degenerate via duplication into coupled structures that are harder to change and prone to inconsistency.

Purely syntactic refactoring greatly improved the program modularity by simply applying the fundamental object-oriented design rule: integrate cohesive data and function abstractions into objects. The refactored program is easier and safer to change. The new requirement can be implemented by a smaller and more focused intervention, without generating any unintended side effects.

In this post the refactoring technique was merely used to change the program structure towards a proper object oriented syntax. There is plenty of room for additional improvement. The next post in the Revisiting Fowler’s Video Store series will explore the refactoring technique at a deeper level.

PM Effectiveness – The Experience Trap

“Experienced project managers can deal more effectively with complex software projects”. Really?

This “conventional truth” is contradicted by INSEAD professors Sengupta and Van Wassenhove’s research on experience-based learning.

They tested the skills of hundreds of professional project managers using a computer-assisted game, built from professor Abdel-Hamid’s renown simulation models for software development. The game was configured to represent a complex environment and most participants confirmed it replicated real-world project situations.

Some of the research results and conclusions are reported in the article “The Experience Trap“, published in the February 2008 edition of HBR magazine:

• experienced managers don´t outperform less experienced ones;
• replaying the game didn´t improve their performance;
• most managers believe project size underestimation and consequent quality, timing and cost problems are recurring flaws in complex projects.

According to the authors, many experienced managers are trapped by their rigid mental models to repeat intuitive and successful behaviours they learned in earlier and simpler projects. The same behaviours can be ineffective or even counter-productive in more dynamic and complex environments.

Research on project management effectiveness is relevant to both individuals, organisations and institutions. Leaders in the professional community, like SAP’s Paul Ritchie, have been highlighting and reflecting on the article main points.

I agree with the colleagues that don’t accept high failure rates nor unsurpassable barriers to continuous learning and professional evolution.

Changes are, of course, necessary at both institutional and individual level. We will address these changes examining the aspects of our professional practice that are most relevant for acquiring proficiency from our experience as project managers:

• How we think. Does our thinking style enables or constrains our perception and learning capabilities?
• How we learn. Do we consciously and actively pursue learning and building knowledge from experience?
• How we lead. What are our focus and purpose for leadership?

Let’s explore these topics on the next posts on Project Management Effectiveness.

The original version of this article was published at Better Projects.

Picture thanks to Włodi@flickr.

The Trust Requirement

Intelligent dialogues require trustful relationships.

In the previous post “Most Requirements are just Design Decisions”, we invited BAs to rethink the rigidity of software requirements.

Relevant environmental changes or new knowledge should trigger corresponding requirements adaptation. Leading this process, a BA adds more business value to their organisations.

Unfortunately, requirements are just requirements for many BAs. They don´t have the opportunity to participate in the intelligent dialogues where strategic design decisions are made.

Effective collaboration between the IT function and other business functions can be a complex and difficult issue in many organisations. This represents a critical credibility problem that must be addressed both by IT executives and IT professionals.

The perceived credibility of an IT professional goes beyond his or her personal competence and integrity; it is grounded in trustful relationships.

Principles

Now I invite you to reflect on your professional relationship style. Is it effective to create and nurture trustful relationships? Here are some principles I have used over the years with good results.

Be a Good Citizen at Workplace. Genuine citizenship behaviour means to me to actively respect all fellow professionals and executives and foster a positive, constructive and productive work climate.

It is also important to have a clear view of what is bad workplace behaviour. David Maister highlights “20 bad workplace habits” from Marshall Goldsmith´s book “What Got You Here Won’t Get You There”.

Reach Out and Touch. Try to see where you and your clients fit in the whole picture. Be open to learn, understand and appreciate their values, interests and the language they use. Then, get closer to them.

Promote Dialogue and Empowerment. I found myself many times eager to present solutions to users as soon as I grasped the most salient problem issues. I learned through experience that rich and frequent dialogues are key elements for more fun and effective processes, that generate better quality solutions and results.

Give Consistent Care. Respond quickly, reliably and honestly to your clients demands. Make sure you consider thoroughly how all your actions will affect their actions. Always be in the front with your clients during major software transitions.

Do these principles resonate to you?

Share with us your thinking about this critical issue for our profession.

I thank Raven Young for indicating David Maister´s post, that inspired me for this post.

This article was originally published at Better Projects.

Picture by Think Panama’s photo stream. Original at Flickr.

Most Requirements are just Design Decisions

The language we use both reflects and influences our thinking.

The term “requirements” has its roots on bureaucratic thinking, that supposes a static and impersonal business world where specialists would be able to uncover, extract and document the definitive specifications for software systems.

Jeff Patton in his post “Requirements considered harmful” alerts us that most requirements are just design decisions. He shows how typical behaviours like asking and giving requirements can be dysfunctional because they justify avoiding responsibility and stop collaboration on a critical success factor for software development projects.

Recognizing that most requirements are just design decisions, gives us a better chance to design and deliver more effective software solutions if we (a) carefully treat them as design products and (b) are ready to adapt them quickly to new circumstances.

Carefully defining requirements as a product of a design process means we should (1) use a collaborative process involving all relevant stakeholders, (2) reflect about the requirements under different perspectives: market, business, users, system and (3) use the most adequate tools and language to express the requirements in a common framework comprehensible to everybody in the team.

Being ready to adapt the requirements means being agile to sense and respond to relevant environmental changes and being open to incorporate new knowledge generated by the organization. The capability to learn and innovate, quickly adapting and creating changes is critical for leadership in the marketplace. Business software should enable these changes and not prevent them.

I know there are situations where requirement changes are beyond our sphere of power and influence. However, as responsible professionals, we should not just let requirement decisions hide under processes, documents and signatures. It is healthy to foster reflection on the what and why of software requirements. As Business Analysts, we should lead this process and help our organizations expand their possibilities.

This article was originally published at Better Projects.

Picture by Lindsey Lissau. Original at Flickr.