Two underrated simple design patterns - "Helper Package" and "Feature Package"

After having dealt with the generic dependencies to the .Net framework and those to specific feature-related .Net or external libraries, I am finally left with Archichect's internal dependencies.^Note

A Foreword

For the roughly 4700 dependencies in my focus I want to write descriptive rules that capture my architectural decisions, so that I can then check the actual code against them. But before I do that, let me make one thing clear: Even though the last few postings used the Archichect tool for formulating and checking rules, I am not so much interested in propagating this specific tool and the notations it provides, but rather in detecting and explaining various concepts that make up this idea of a useful, yet precise language for handling software architectures.

Just like notation, tooling is something that must provide much more than only "the right functionality": At the least, it must tie in with existing infrastructures (a tool that runs only on Windows; or only with Microsoft's TFS, is not very helpful in a Unix development environment). But, moreover, it seems that it must also appeal to its users, and this typically requires the build-up of a "fan community" which needs to like the principles and the concrete implementation (including all its bugs and hiccups and inadvertent effects) and, last but not least, also itself, i.e., the community—how its members communicate among themselves about the open problems, the evolution path to follow, and how the problems and its resolution alternatives are perceived. I am, right now, not really prepared to act a great deal in these areas—and would therefore also encourage you to take over anything you might have learned from my ponderings to the toolset of your liking: Modern tools are typically quite open, with configurable models and all sorts of scripting or even plugin capabilities, so hopefully you can add all you like also there.

On another level, my "relativism" about the "right tooling" also means that in each of my postings, I want to present a concept that is of more general interest than whether Archichect has—or lacks—features supporting that concept. I hope that each text makes it clear enough where this boundary between the interesting problem or solution, on the one hand, and the workings of Archichect or any other tool, on the other hand, is located.

Those patterns ...

Ok then. After the preliminaries above, let me write in this posting about two design patterns: Namely ways to separate out a sub-package from a large package. I need this to explain the intended static architecture of Archichect (and other programs) more succinctly, so that I can then explain, in another step, some more prescriptive rules that will capture many of those dependencies that still look like "violations".

Most probably, the following patterns that describe how, and why, packages can be subdivided have already been described somewhere else. However, I did not find anything on the internet or in my (few) books, except a short paragraph on Martin Fowler's refactoring.com by Gerard M. Davison, with an additional note by Fowler himself—but both only describe it in terms of a refactoring and not as a group of patterns. So let me present my descriptions of the two patterns I have identified, using a shortened GoF-like description template.

Pattern name & classification: Helper Subpackage (or Helper Package), a structural pattern

Intent:

When a package becomes too large, split off some part into a new subsidiary package.

Motivation, forces, scenario:

A package "search" for searching in some data set contains a set of search controllers (e.g. for selecting various sets to search, various outputs for receiving search results) and also genuine algorithm classes, e.g. for field-wise equality or unequality matching of search parameters to the searched objects. Over time, the package is enhanced with more and more refined and complex algorithms, e.g. some variant of regular expressions, or fuzzy matching. The number of all classes—search controllers as well as algorithms—in the package increases to a point where it becomes hard to separate the controllers (and possibly other groups of classes) from the algorithm classes; and where it becomes hard to manage the dependencies between the controllers and the algorithms. Both these problems can be reduced by moving the algorithm classes into a subpackage, say "search.algorithms", and then enforcing and maintaining that controllers may use classes from the algorithms package, but not vice versa.

Structure & participants:

• main package
• helper package
• external API users

Instead of having a single package, move "more internal stuff" into a helper package. The pattern allows for the possibility that some of the helper classes are part of a public API: In such a case, the external API caller will access both the main and the helper package.

Consequences:

• The number of classes per package is reduced.
• Classes are sorted into named "drawers"; only the main "drawer" does not get a special qualifying name.
• If the helper classes access resources common with the main classes, a cyclic dependency from the helper package to the main package will occur (see the discussion on "practical cycles" below).
• If external API callers should not access the helper package, a Facade may encapsulate both the main and the helper classes.

Known Uses:

JDK's javax.swing.border (some more explanations are necessary here)

Related Patterns:

In the Helper Package pattern, the single or main dependency is from the main package to the helper package: "main uses helper". In the Feature Package pattern, on the other hand, the majority of the dependencies is from the feature subpackages to the main package: "the features use the (common) main algorithm base". This difference is the main rationale to distinguish the two patterns: If it is not clear why a sub-package is extracted from a main-package, both purposes might overlap, resulting in a tangle of cycles between sub-package(s) and main package which may be hard to clean up or even understand later.

So much for the first pattern. The second one is:

Pattern name & classification: Feature Subpackage (or Feature Package), a structural pattern

Intent:

When a package becomes too large, split off some feature-related parts into new packages, with one sub-package per feature.

Motivation, forces, scenario:

A package "parallelcollections" exposes multiple thread-safe collection types, e.g. bags and dictionaries. The number of common and type-specific classes in the package is so large that recognizing which class uses which other classes becomes difficult if all classes resided in the same package. Thus, a separate sub-package is provided for each collection type, e.g. "parallelcollections.bag", "parallelcollections.dictionary" and so on. The classes in each such feature package may access common algorithms in the main package, whereas the main package typically does not access the feature packages.

Structure & participants:

• main package
• feature packages
• external API users

Instead of having a single package, move stuff related to some feature into a corresponding feature package. Typically, the external API users primarily access the feature packages.
The pattern allows for the possibility that some of the classes in the main package are also part of the public API: In such a case, the external API caller will access both the feature packages and the main package.

Consequences:

• The number of classes per package is reduced.
• Classes are sorted into named "drawers"; only the supporting main "drawer" does not get a special qualifying name.
• If the main package contains references to the feature packages (e.g. for a  factory that creates various feature class instances), cyclic dependencies between the feature packages and the main package may occur (see the discussion on "practical cycles" below).
• If external API callers should not access the main package, each feature together with some part form the common API may be encapsulated behind a Facade.

Known Uses:

JDK's javax.swing.table, .Net's System.Collections.Generic (also this needs elaboration...)

Related Patterns:

Helper Subpackage; see there for the main difference between Feature and Helper Package.

These two patterns are certainly not rocket science. But I think it's worthwhile to point out the fundamental difference between the "helper" and the "feature" approach. The main reason why I think it's useful to have these two concepts is described above in the "Related Patterns" paragraph of the "Helper Package" pattern: If classes were moved to a single sub-package for both reasons at the same time—factor out "a common helper" and also "separate features"—, you will probably get a considerable volume of both "up" and "down" dependencies, which tightly couple the packages together. Thus, important additional benefits of separating out the package, like improved testability, are more or less lost. To prevent this, deciding on exactly one of the two patterns seems useful.

Practical cycles

The desire for cyclefreeness nonwithstanding, I think (and see in examples) that, for pragmatic reasons, a few cyclic dependencies might not be that harmful. I would call such cases "shackled", as in "shackled helper package" and "shackled feature package": In both cases, one is not really free to use the referenced package on its own; but as the package came from a larger one anyway, and its primary cause was not decoupling, but organizing a large set, this is not a disadvantage a priori. Still, you have to deal with these cycles, which will make testing more difficult in parts, and reuse on the whole. Here is a discussion what you can do with the cycles of a "shackled helper package"; for "shackled feature packages", similar considerations can be made:

a) You backtrack, don't extract the subpackage and continue to live with the monolith. Of course, you "got rid" of the cycle only on the face of it, because the class dependencies have not changed a bit—but it might look good to your management or even your architect. This is most probably not acceptable—therefore, continue with option b).

b) You use some standard method for cycle elimination, e.g., interface extraction and maybe an abstract factory.
Pros: No cycles.
Cons: Added complexity that does not seem to have any benefit - after all, you just extracted the package to organize your code better, not to introduce new abstractions. If this is not acceptable, go to c).

c) You analyze the original package to find out whether you can split it to remove the cycle. Quite often, classes in a package have an internal hierarchy akin to the following:

Typically, the "worker part" is the heavy-weight part, which you (completely or partly) want to extract into the new package. But this creates that ominous cycle:

However, the diagram hints at what can be done to remove the cycle: Split the old package into a "lower utility" and an "upper facade" part, and live happily ever thereafter:

Pros: No cycles, no added complexity on class level.
Cons: Three packages; this may be especially troublesome if both the "utility" and the "facade" parts contain APIs that have been exposed to the user, who expects these related classes to remain in a single package. If your package structure is part of the API (e.g. because you use .Net namespaces as your package structure), this will not work out. What about option d)?

d) You might be able to map classes to different packages without a change to the external API. The most flexible way is, of course, to have an explicit mapping table. With Archichect rules, you might for example write:

INTEGRATION := mainpackage:(IntegrationClass1|IntegrationClass2|IntegrationClass3)
UTIL        := mainpackage:(UtilClass1|UtilClass2|UtilClass3)

with cyclefree and clearly understandable rules

INTEGRATION        ---> mainpackage.helper
mainpackage.helper ---> UTIL

Pros: No cycles on package level, no added complexity on class level.
Cons: Alternate, specialized package mapping. If you are not happy about this, i.e., you always want (or have to) map packages to your language's publicly visible name groups (like Java packages or .Net namespaces), go to option e).

e) You live with the cycle.
After all, it's only an internal detail of exactly this module. This is not something to lose sleepless nights about.
Pros: You solved the problem in the most straightforward fashion.
Cons: ... but that cycle, oh that cycle ...

To sum it up succinctly:

• Helper package = sub-package into which the main package looks down
• Feature packages = sub-packages with features which themselves look up into the main package

With these concepts in mind, let me describe Archichect's intended package architecture, and the rules to describe it, in the next posting.

Note: To be precise, I am considering here only direct, static, namespace dependencies: In other words, I am currently not interested in

• indirect dependencies (e.g. which classes implement which interfaces indirectly),
• dynamic dependencies (e.g., which objects create or have communication channels to which other objects, and when),
• other static dependencies (e.g. .Net assembly dependencies).

One step at a time! Maybe I have some ideas about some of these dependencies later.