In an internal Number Six blog, Clayton Neff and Abby Fichtner recently brought up some good points about how essential good object-oriented fundamentals are to iterative and incremental development (Agile or not). I think the more a team embraces change and organizes their development approaches around making rapid, high quality changes to their software, the more the benefits of good OO design become critical to their long-term success. You have to continually pay down that technical debt on any high-speed team (again, agile or not), and loosely-coupled, highly-cohesive designs are part of that velocity.

A lot of companies we talk to are looking for a silver bullet tool that will magically transform their Fresh-Out-of-College “Java Experts” or their “COBOL Gray Beards” into Rock Star Ninja Developers. They’ll pay astonishing amounts of money for these tools - four, five, six thousand dollars a seat. But they won’t send the whole team to training together, and invest that money in peoples’ hard skills. “Send Bob, he’ll teach the rest of us.”

On truly Agile teams, or teams adopting more agile practices, the technical approaches involved in short-cycle iterative and incremental development aren’t going to solve the basic problem of poor developer skills, particularly with OOA&D. In fact, what a two week iteration that produces production quality software will do is point out your team’s weeknesses, EVERY DAY, EVERY WEEK, ON AND ON AND ON until you fix them. In some organizations, the disease may be easier for teams to live with than the cure. Some agile adoptions will likely peter out simply because it was too painful to continually have their process point out that, well, they’re not very good at writing software. It would be much more comfortable to just make lots of documents and spend a lot of time controlling change — because figuring out how to adapt with crappy code is just too hard, and they’re tired of Frequently Failing Fast.

Rejecters will says “Agile didn’t work for us” (or “iterative” or “RUP”). And of course, they’re right. But in this case the Process failure is the компютри втора употребаeffect, not the cause. It’s poor fundamental hard skills (like OOA&D knowledge) that made it impossible for the team to produce quality software every few weeks, and steadily change it over time. The Process simply EXPOSED these failings.

What do you think?

It hit the desk with a thud. “What’s that?” “Oh, that’s the Risk Management Plan.” “Great, let’s see the Risk List?” “Oh, we didn’t bring that, it’s on Tony’s laptop somewhere.”

This was a real conversation. We were meeting with some of the staff in the PMO and they had brought their Risk Management Plan for review. It was an impressive document with 65 pages of everything you ever needed to know about risk management. This was the culmination of weeks and weeks of effort and many meetings. The sad thing is that in the time they had spent documenting the process they could have collected, analyzed and responded to all the program’s risks and been in a much better position to ensure program success.

You don’t need to spend weeks and weeks defining a risk management process, it has already been documented to the finest level of detail and there’s not a lot of options to ponder. Likewise, you don’t need to spend hundreds of thousands of dollars to be told that you need a change control board (sadly another true story).

As real process engineers will tell you, a process definition is not a process. Instead of spending months and months documenting processes it is far more effective to reference existing documentation. Keep the process simple and transparent so that it can be easily implemented and iterate often in the early stages to right size the process to fit the team. Remember a process definition is not read but referenced, so make sure it is small, online and reference-able.

What is ClearQuest’s LDAP setup trying to do? It’s a question I’ve asked myself a million times. This blog entry is an attempt to sketch out my assumptions. Follow along and see whether I’m whacked out or what.

LDAP Lookup, Not Management

The first thing that threw me was that ClearQuest uses LDAP only to do a password lookup. It doesn’t use any of the rest of the information for anything at all. Further, being in LDAP alone isn’t enough to get you into ClearQuest or to update your email address. Why is this? Probably because that lets us do two things: 1) mix LDAP and non-LDAP users and 2) continue to manage groups inside of ClearQuest.

What’s In a Name?

The second hurdle to get over was that we can set up this validation so that the thing the user types in to gain access to the system does not have to match what is in the “Login” field of the ClearQuest user management tool. That’s really hard to get the noggin around, but it’s kind of neat when you do, because ClearQuest uses that “Login” field to indicate the user’s primary key. Whatever you use as your naming structure there is what shows up in the drop-down box for user reference fields.

This is exciting, because it means that we can start putting human-readable information in the “Login” field but allow the users to use their network logins. So, for example, I can log in with jmxm253, but others can pick john_martin from the drop-down.

More Indirection

The third hurdle was tougher, probably because I don’t see a great use for it. It seems that ClearQuest will let you do two validation checks.

When you configure ClearQuest to use LDAP, you take three important steps:

1. Create a connection method/password/etc for getting into the LDAP provider (e.g., a username to log into Active Directory and browse around)
   • (installutil setldapinit)
2. Create a search string for finding the user who’s trying to log in
   • (installutil setldapsearch)
3. Create a mapping to connect a ClearQuest user field (like login_name or email) to an LDAP field (like sAMAccountName or mail).
   • (installutil setcqldapmap)

The really weird part that throws you for a loop, IMHO, is that #2 and #3 don’t have to be related! The example provided with the instructions seems to imply that if you set your search criteria in #2 to be sAMAccount=%login%, then you also must set the mapping up so that CQ_LOGIN_NAME maps to sAMAccountName. And I wracked my brain to figure out why you’d have to do #3 if you’re doing #2.

This is the important bit: %login% is what the user types into the login box when he/she launches ClearQuest. %login% doesn’t have to be the thing the administrator puts into login_name in the CQ user tool or in any other field.

Here’s the flow of events when an LDAP-enabled user logs in:

1. User launches ClearQuest for Windows.
2. User types in a User Name. The User Name will match whatever you set equal to %login% in STEP 2 above. If you said something like (sAMAccountName=%login%), then the user types in his/her network login ID. If you said something like (mail=%login%), then the user types in the email address that Active Directory has.
3. User puts in a password and hits OK.
4. ClearQuest connects to LDAP using what you put in #1 above.
5. ClearQuest searches LDAP for this particular user based on the search criteria you put into #2. (So it’s checking the user exists and is in the right place before checking the password.)
6. When ClearQuest finds that user, it checks to see if the password is correct.
7. If the password is correct, ClearQuest compares the LDAP user’s field to the appropriate user record field in ClearQuest as you mapped in STEP 3.
8. When the user passes this and is logged on, he/she finds a record and sets himself/herself as the owner. What does he pick on the drop-down list? The ClearQuest logon, which doesn’t have to look like anything already entered.

So, I don’t know why you would do this, but you could, for example, have the user log in using his/her network ID and then have CQ verify that the email address it is storing matches the one LDAP is storing.

Example

Say you have a CQ user called “john_martin” (currently user login_name). You want him to login using his network ID (jmxm253) and have CQ validate that the email address we have (test@test.com) is correct. If we wanted to make sure that jm253 could only log in if he’s in the ‘cq users’ group, you would make the following settings from the command line (this is also a good example of handling spaces):

1> installutil setldapinit mydbset admin “” “-h myprovider.mycompany.com -p 389 -D ‘CN=Service Account Five,DC=myprovider,DC=mycompany,DC=com’ -w SECRETWORD!”

2> installutil setldapsearch mydbset admin “” -s sub -b ‘OU=my users,DC=myprovider,DC=mycompany,DC=com’ ‘(&(objectCategory=person)(sAMAccountName=%login%)(memberOf=CN=cq users,OU=Security Groups,DC=myprovider,DC=mycompany,DC=com))’

3> installutil setcqldapmap mydbset admin “” CQ_EMAIL mail

Now, he’ll type in jm253/password to log in (CQ will see if that is his account name in LDAP and if he’s in the “my users” LDAP group), ClearQuest will make sure that LDAP’s email matches the one in ClearQuest, and any report of users (or drop-down field like owner) will use john_martin as the key. History, for example, will show that even though he used jmxm253 to log in, john_martin is the thing listed as the actor.

Also, he won’t be able to use john_martin to log into ClearQuest and you’ll have no idea what his network ID is. Do you care? If you care, but you’d like the unique user key to continue to be john_martin, then you can make CQ_MISC_INFO as the mapped field and map it to sAMAccountName.

There. I’ve used my own name about a million times. Thought ought to make me famous. Did it make the issue clearer or more muddy?

I’ve really been enjoying the book Agile Project Mgmt w/ Scrum in preparation for the upcoming Scrum Master class. Awhile back I wrote an internal Number Six blog on “Agile & Fixed Price Contracts” which generated some interesting discussion (thanks Clayton, Joe, & Nate) along this thread. As I was reading the book last night, I noticed that Ken Schwaber dedicated a short Appendix (D) to the topic of bidding Scrum on fixed-price, fixed-date contracts.

<tangent> For those of us that have written many proposals, we know how grueling it can be. I’ve participated in proposal efforts that range from the good to the bad to the very ugly. I’ll defer some of the interesting organizational dynamics of proposal teams to another blog. The actual work on any given proposal can range from an organized, focused development of a themed offering to a fire drill where the team scrambles from start to finish, copying and pasting excerpts from disparate artifacts scattered all over the place and then hand delivering the polished turd within an hour of the deadline to the FedEx drop box or client site. </tangent>

In Ken’s Appendix, he calls out an interesting advantage for making Scrum part of an RFP Response, the Product Backlog. The Product Backlog is a somewhat unique way to communicate to the client that not only the breadth of requirements are understood, but also their prioritization. We all know the customer doesn’t understand all the requirements, and we also know that even if they did they would change them as the project moves forward. So why not pitch the customer’s ability/flexibility to change the requirements in the proposal? Obviously, this only works if the product is released incrementally (not unique to Scrum) rather than in a big-bang at the end. Once the high-priority requirements are deployed in working, incremental releases, it’s a win-win to tell the customer they can steer the project in whatever direction they think makes the most business sense.

Giving the customer this flexibility (that they already have, ironically) will make them feel empowered and will often differentiate a proposal from the crowd. One of the biggest fears of bidding companies is the degree to which the customer is likely to change the requirements mid-stream. In my experience, most proposals bake significant buffers into them specifically to absorb inevitable scope creep. So this will also have a bid price/cost impact as well.

One of things that attracted me to Number Six and has kept me engaged as well is a desire to have an open and ‘learning’ culture. Our Communities of Practice (COPs) were created in large part to try and give people an outlet to dialog and learn about things they’re personally interested/passionate about.

In the last Mgmt COP call, I mentioned a book that Ken Clyne recently recommended to the Office of Technology called The Fifth Discipline - The Art and Practice of the Learning Organization by Peter M. Senge. I’ve been thoroughly enjoying it and would highly recommend it. One of the concepts discussed in this book is that of how to build a ‘learning-oriented culture’ in your organization. One might ask… What is a learning culture? Why should I care? How do we start building it?

The psychological definition of learning is “the modification of behavior through practice, training, or experience”. There has been and continues to be a massive amount of research being done in the area of how complex organisms learn (e.g., psychology, linguistics, neuro\- and cognitive sciences). The study of complex living systems has yielded an understanding that they/we largely operate based on local sensing and action. Peter Senge gives an example of this:

Whatever “centralized” control does exist in nature is possible precisely because of complex networks of local control. We have no idea how to walk, but once this “body knowledge” is developed, the body responds to our conscious directives; without that body knowledge, all the central directives in the world would be ineffectual.

Applying this view of learning to an organization can reveal a lot about how we work together and the nature of our organization as a whole. Why should we care? The more we’re able to learn, both individually and as a group, the more likely we will be to produce the results we really want to produce (i.e, meet our objectives, realize our vision, etc).

So how can we encourage practical learning in our organization? Step one is each of us caring about our personal development. Peter mentions that we shouldn’t view learning as an “add-on” to our “regular work”. Learning and personal development shouldn’t be viewed as a program or initiative, and it certainly should not be thought of in terms of formal training only. Peter’s advice would be for each of us to really reflect on what we personally care and are passionate about.

In his presentation on Implementing RUP - Agile Style Per Kroll talks about organizing processes and practices along two dimensions of a process map in order to compare them and analyze which are more suitable for your project or organization.The Y axis depicts the degree of iterative development. At one end are waterfall processes characterized by projects with late integration and testing and carrying few risks.

At the other end we have iterative processes that are risk driven and feature continuous integration and testing. Typically RUP projects would be expected to map to the lower quadrants of the process map.The X axis depicts the level of ceremony. Ceremony is a term often used in the Agile community. The following definition from dictionary.com seems apropos:

cer.e.mo- ny

1. A formal act or set of acts performed as prescribed by ritual or custom: a wedding ceremony; the Japanese tea ceremony.
2. A conventional social gesture or act of courtesy: the ceremony of shaking hands when introduced.
3. A formal act without intrinsic purpose; an empty form: ignored the ceremony of asking for comments from other committee members.
4. Strict observance of formalities or etiquette: The head of state was welcomed with full ceremony.

Agile projects would translate to the lower-left quadrant of the process map. Although some Agilists balk at the extent of the detailed guidance RUP is of course a framework and fits well with an Agile adoption (see Skinnier RUP by Scott Ambler for a lengthier discourse).

Core Principle’s of RUP

Agile Principles

There are a lot of similarities between the two schools of thought. For me the most striking similarity though is the emphasis on working software as the measure of progress. RUP is very clear that each iteration (after inception) concludes with an executable release and that a key deliverable of
elaboration is an executable architecture. Granted RUP places a lot of emphasis on the Vision document and the Use-Case model but these artifacts are not (from our definition above) “formalities without an intrinsic purpose” they exist for the express purpose of verifying concurrence with stakeholders and communicating that decision to the team. One of the key deliverables in RUP is the problem statement (typically embedded in the Vision document) which could easily fit on a business card and if you’re an agilist, this has to resonate.So, a RUP project very clearly belongs in the lower quadrant perhaps tending further down and to the left as agile practices become more widely accepted. What is striking though is that we come across many projects in the top-right quadrant. Not directly at our clients but indirectly at events and notably when we interview people. The dialog often goes somewhat like this. “So, I see you been using RUP for the last 3 years, how would you rate your expertise with RUP”. “Probably around a 8 or 9″. “I see, could you explain RUP in a nutshell for me?”. “Certainly, RUP provides a wealth of document templates that help us get started quickly on our project deliverables”. This is not an integrity issue or an education issue, people like this do know the process they’ve been using for years, it just isn’t RUP. Maybe they started out with the best intentions but just like that puppy or kitten that arrives at Christmas and then it sinks in that this bundle of fun needs care and feeding then it gets abandoned. If you or your client fits into this category then maybe it’s time to be reintroduced to RUP and start implementing it as it was intended, down and to the left.

I recently participated in a Scrum Master certification class at Number Six Software. The class was interactive and fun, and I especially enjoyed the group discussions prompted by questions from fellow classmates.

 

A couple interesting ideas were discussed pertaining to the beginning of a Scrum project, “Requirements and Architectural Runway” and “Interation 0″.

A requirements and architectural runway is a concept mentioned in some of Mike Cohn’s recent writings including this interesting slide deck (see slides 27-30). The idea refers to priming the requirements and architecture pump a bit before starting the first (or each) Sprint with the goal of minimizing the impact of poorly defined or unrefined requirements and architecture on the Sprint’s velocity. Mike implies this is needed to “assure you’re building the right code” and I can see how it would potentially optimize the designers and developers time out of the gate in the Sprint, as well as mitigate delays in getting time from SMEs and stakeholders to clarify requirements. This is not an attempt to do all the requirements up front (we all know that doesn’t work), and Mike says that the size of your runway will vary depending on the project.

So why not do a requirements and architectural runway as Mike suggests? Our instructor (Jim York) was quite opposed to the idea. When any of us kickoff a project, we typically have a series of JAD sessions or requirements workshops to do things like establish the vision, the problem statement, the high-level requirements, etc. This is fine in Scrum according to Jim. My understanding of his beef with a runway is that it takes the requirements analysts somewhat outside the Sprint and team, which does serialize the process a bit. This somewhat undermines the Scrum concept of the Sprint’s being self-contained and the whole team ‘owning’ the whole Sprint Backlog. He said the recommended Product Backlog size to start a project is about 1.5 times the size of what can roughly be done in a single Sprint. However, the Product Backlog contains higher-level requirements and stakeholder requests, not typically refined requirements.

Secondly, the concept of an “Iteration 0″ was discussed in the context of having a Sprint or partial Sprint to start the project that allowed the team to set up the bare minimum infrastructure (e.g., test harnesses, tool setup/installation, preliminary architectural plumbing, etc.) for the project. I personally really like this idea and think it keeps the Sprint 1 Backlog cleaner for the Product Owner who doesn’t necessarily care about or see the value in these types of activities. If you buy into the idea of a requirements runway, an Iteration 0 would provide time to develop that as well.

On the first day of Agile 2007 I attended a Q&A session with Bob Martin, Hubert Smits and Mary Poppendieck which was a follow-up to a series of beginner sessions on XP, Scrum and Lean.

During Bob’s session on XP he emphasized low-ceremony tools and had some fun at the expense of projects that spend money on requirements management tools. This was in stark contrast to him standing there with the entire feature set in his hand on a set of cards. He created an iteration plan by dropping the cards into piles on the floor: this iteration, next iteration, the one after that and then all the rest. How about version control? Bob said use a shoe box and if someone loses the box then just recreate the cards and anything forgotten wasn’t worth remembering anyway.

Bob also talked about the whole team principle in XP and the necessity of all the contributors to an XP project sitting together.

Afterwards during the Q&A session someone asked about how such techniques scale to support distributed development. Great question I had been pondering that myself. Bob was fairly blunt and stated that team members need to sit together. Hubert and Mary handled the question with a bit more grace but didn’t offer up much of a solution.

I though this rigid stance peculiar for two reasons:

1. The Agile Manifesto states “Individuals and interactions over processes and tools”. For many and varied reasons (for example quality of life and concern for the environment) we are now affording individuals the flexibility and means to work remotely. So, why in this one instance are the needs of the individual placed after the principles of a process.
2. At the conference, there were many exhibitors demo’ing Agile tools that enable distributed teams. Two I found particulraly intriguing were Rally and Thought Works. There is obviously some investment so, this must be a problem worth solving.

Perhaps the next big challenge for the Agile community is to set aside some of its principles for the sake of making a much bigger difference in our development environment (and I’m not talking about an IDE).

I went to the DC SPIN (Software Process Improvement Network) meeting tonight with guest lecturer, our very own, Nate Oster. The topic was OpenUP. Not surprisingly, Nate did a superb job presenting the material…very knowledgable, very energetic, an excellent representitive for Number Six. We also had a good showing of Sixers in attendance.

A sampling of some of the interesting discussion points from the meeting…

First, Nate distinguished OpenUP from agile methods such as Scrum & XP, describing it as a “minimal”, agile, open, version of RUP. If I were to guess, most people who know anything about agile methods probably think of them as minimal, but not always in a good sense, or the intended sense. The idea conveyed tonight was that many RUP projects begin with the full set of artifacts and decide which ones to remove, whereas OpenUP looks at this from the other direction…start with nothing and add the necessary artifacts. While one could argue that this is a matter of semantics (i.e., who cares which end you start at as long as you end up at the right level), I think it establishes a good mindset for the team…only necessary/important artifacts should be created and maintained. There are likely to be disagreements initially about the “right” level of documentation, but if there are enough retrospective points in the process where artifact usefulness is assessed, the right level will emerge over the course of the project.

There was also some interesting discussion about the way features/stories/requirements get prioritized. Nate clarified what I think is a critical, oft-overlooked nuance of the The Planning Game . I’ve heard more than one agilist state plainly that the feature cards are ‘prioritized by the customer’. This is not correct as it misses a key aspect to the feature prioritization process. Nate rightly stressed the collaborative nature of this effort and that the architect’s input regarding architectural dependencies and risk are weighed with the customer’s business priority to establish the prioritization. This architecture-centric view, this focus on eliminating risk through iterative elaboration, is something that OpenUP rightfully kept from it’s passé parent, RUP. I was very happy to hear that in OpenUP, like RUP, not all iterations are created equal (i.e, IECT is still in play over the course of the project).

So you are working on a project and you have two object and want to know if according to the business they are the same item. So you call .equals. It returns false because they aren’t the same object in memory. So you override the .equals method to compare the attributes that make the items equal to the business. This opens a can of worms. .equals is used by the system for many things and overriding it shouldn’t be taken lightly. If you have no reason to make HashMaps consider the items the same key then you probably don’t want to override .equals. Make a new method like .isTheSameAs or something that makes sense to the business.
I have seen many a good programmer implement a bad .equals in Java. So what makes a good implementation of .equals? Well first it has to meet the following contract.

1. The hashCode method must return the same integer value every time it is invoked on the same object during the entire execution of a Java application or applet. It need not return the same value for different runs of an application or applet. The Java 2 platform (Java 2) documentation further allows the hashCode value to change if the information used in the equals method changes.
2. If two objects are equal according to the equals method, they must return the same value from hashCode.
3. The equals method is reflexive, which means that an object is equal to itself: x.equals( x ) should return true.
4. The equals method is symmetric: If x.equals( y ) returns true, then y.equals( x ) should return true also..
5. The equals method is transitive: If x.equals( y ) returns true and y.equals( z ) returns true, then x.equals( z) should return true.
6. The equals method is consistent. x.equals( y ) should consistently return either true or false. The Java 2 javadoc clarifies that the result of x.equals( y ) can change if the information used in the equals comparisons change..
7. Finally, x.equals(null) should return false.

Most experienced programmers know the rules of .equals and .hashCode and can implement a something that meets the Java contract but still causes problems. In an attempt to make things better the Eclipse IDE has a code assist that will generate a method that implements a .equals and a .hashCode that meet the contract. This makes the danger worse because programmers feel confident the implementation is safe.

Just because a .equals and .hashCode implementation meets the contract doesn’t mean it is safe. If the attributes used in the .equals and .hashCode are mutable then the hashCode can, and should according to the contract, change. Why is this so bad? If the object is a key in a HashMap or HashTable and that object changes after being used as a key the value it references is lost in the HashMap or HashTable.

Consider this example which uses the Eclipse generated .equals and .hashCode.

package evil.equals;public class Name {
 private String firstName;
 private String lastName;
 private String middleInitial;
 @Override
 public boolean equals(Object obj) {
  if (this == obj)
  return true;
  if (obj == null)
  return false;
  if (getClass() != obj.getClass())
  return false;
  final Name other = (Name) obj;
  if (firstName == null) {
  if (other.firstName != null)
  return false;
  } else if (!firstName.equals(other.firstName))
  return false;
  if (lastName == null) {
  if (other.lastName != null)
  return false;
  } else if (!lastName.equals(other.lastName))
  return false;
  if (middleInitial == null) {
  if (other.middleInitial != null)
  return false;
  } else if (!middleInitial.equals(other.middleInitial))
  return false;
  return true;
 }
 public String getFirstName() {
  return firstName;
 }
 public String getLastName() {
  return lastName;
 }
 public String getMiddleInitial() {
  return middleInitial;
 }
 @Override
 public int hashCode() {
  final int prime = 31;
  int result = 1;
  result = prime * result
  + ((firstName == null) ? 0 : firstName.hashCode());
  result = prime * result
  + ((lastName == null) ? 0 : lastName.hashCode());
  result = prime * result
  + ((middleInitial == null) ? 0 : middleInitial.hashCode());
  return result;
 }
 public void setFirstName(String firstName) {
  this.firstName = firstName;
 }
 public void setLastName(String lastName) {
  this.lastName = lastName;
 }
 public void setMiddleInitial(String middleInitial) {
  this.middleInitial = middleInitial;
 }}package evil.equals;import java.util.Hashtable;public class NameTest extends Name {

hash:603748753
hash:-455745749
not found

You can see that the value “value” is now stranded in the HashTable. This is something to think about before you override .equals. You may also be thinking well I am not going to use the class as a key in a Hash but objects are often used as keys in maps my persistence layers or other architectural constructs you may not be aware of.

My advice is unless you make an object immutable think twice or thrice before you implement a .equals.