Tag Archives: architecture

software

Applied Cosmology: Machian Dynamics

Leave a comment

Julian Barbour wrote the book titled “The End of Time: The Next Revolution in Physics” [http://www.platonia.com/ideas.html]. He explains how our failure to unify General Relativity with Quantum Theory is because of our ill-conceived preoccupation with time as a necessary component of such a theory. According to Machian Dynamics, a proper description of reality is composed of the relationships between real things, not a description with respect to an imaginary background (space and time). Therefore, all you have is a configuration of things, which undergoes a change in arrangement. The path through this configuration-space is what we perceive as the flow of time.

We apply this very model of the universe in configuration management.

Software release management is a configuration management problem. The things in configuration-space are source files. A path through configuration space captures the versions of these source files relative to each other as releases of software are built. Our notion of time is with respect to these software releases.

Enterprise resource management in the communications industry involves many configuration management problems in various domains. We normally refer to such applications as Operations Support Systems.

In network resource management, the configuration-space includes devices and other resources in the network, their connectivity, and the metadata (what is normally called a “device configuration” which needs to be avoided in the context of this discussion for obvious reasons) associated with that connectivity arrangement.

In service resource management, the configuration-space includes services, their resource allocations, and the subscription metadata (what is normally called a “service configuration” which needs to be avoided in the context of this discussion for obvious reasons) or “design”.

Such applications have a notion of configuration-space, because such systems cannot operate in a world that is limited to its dependence on a background of space and time. We need to be able to travel backward and forward in time arbitrarily to see how the world looked in the past from the perspective of a particular transaction. These applications enable users to hypothesize many possible futures. Perhaps only one of which is brought into reality through a rigorous process of analysis, design, planning, procurement, construction, and project management. Reality is always from the perspective of the observer, and one’s frame of reference is always somewhere on the path in configuration-space.

software

Software engineering is applied cosmology

1 Comment

Engineering is applied science. Some people believe that software engineering is applied computer science. In a limited sense, it is. But software is not entirely separated from hardware. Applications are not entirely separated from processes. Systems are not entirely separated from enterprises. Corporations are not entirely separated from markets. For this reason, I believe what we do is not software engineering at all. It is not limited to applied computer science. Our engineering discipline is actually applied cosmology.

software

designs are useless – like planning

Leave a comment

“In preparing for battle I have always found that plans are useless, but planning is indispensable.” -Eisenhower (from Planning Extreme Programming)

I believe this is true, because “no plan survives contact with the enemy”. In software, the enemy is in the form of dark spirits hidden within the code, both legacy and yet to be written. Because plans (the schedule of work to be done) are intimately tied to designs (models of the software), it must also be true that no design survives contact with the enemy. Any programmer who begins writing code based on a preconceived design will almost immediately feel the pain of opposing forces that beg to be resolved through refactoring; programmers who lack this emotional connection with their code are probably experiencing a failure of imagination (to improve the design).

Therefore, I think we can return to the original quote and state its corollary: designs are useless, but designing is indispensable.

All this is to say that for the above reasons, I think these ideas strongly support the notion that the process artifacts (e.g., Functional Solution Approach, Functional Design, Technical Design) are more or less useless, because as soon as they are written they are obsoleted by improvements that are discovered contemporaneously with each line of code written, but the act of producing them (the thought invested in designing) is indispensable to producing good software.

This leads me to conclude that we should not fuss so much about the actual content of the artifacts, so long as they capture the essence to show a fruitful journey through designing—that problems have been thought through and decisions have been based on good reasoning. Worrying about the content being perfectly precise, comprehensive, and consistent ends up being a waste of effort, since the unrelenting act of designing will have already moved beyond the snapshot captured in the artifact.

Coincidentally, this theme also aligns with the notion of a learning organization espoused by Lean. The value of designing is the facilitation of learning.

software

transparent persistence

Leave a comment

Advantages

Transparent persistence has emerged into the mainstream over the past few years with the popularity of JDO and JPA for enterprise application development. This approach offers the following advantages.

  1. Domain modeling is expressed naturally as plain old Java objects (POJOs) without having to program any of the SQL or JDBC calls that are traditionally coded by hand.
  2. Navigation through relationships – objects are naturally related through references, and navigating a relationship will automatically load the related object on demand.
  3. Modified objects are stored automatically when the transaction is committed.
  4. Persistence by reachability – related objects are automatically stored, if they are reachable from another persistent object.

The programming model is improved by eliminating the tedium that is traditionally associated with object persistence. Loading, storing, and querying are all expressed in terms of the Java class and field names, as opposed to the physical schema names. The programmer is largely insulated from the impedance mismatch between Java objects and the relational database. The software can be expressed purely in terms of the domain model, as represented by Java objects.

Challenges

When developing domain objects, persistence is only one aspect. The business logic that applies to the graph of related objects is the most important concern. Transparent persistence introduces challenges to executing business logic to enforce constraints and complex business rules when creating, updating, and deleting persistent objects through reachability.

For example, an equipment rental application may need to enforce the following constraints:

  • When creating equipment, it must be related to a location.
  • When creating a rental, it must be related to a customer, and ensure that the equipment is available for the duration of the rental.
  • When updating a rental, it must ensure that equipment is available for the duration of the rental.

JPA 2.0 does not provide sufficient mechanisms for enforcing these constraints, when creating or updating these entities through reachability. The responsibility is placed on a service object to manage these graphs of entities. The constraint checking must be enforced by the service object per transaction. Java EE 5 does not provide any assistance to ensure that the constraint checks (implemented in Java) are deferred until commit, so that they are not repeated, when performing a sequence of operations in the same transaction.

Adding a preCommit event to a persistent object would provide a good place for expressing constraints. Allowing this event to be deferred until transaction commit would provide the proper optimization for good performance. Of course, preCommit would need to prevent any further modifications to the persistent objects enlisted in the transaction. This would factor out many of the invariants so that they are expressed per entity, removing the responsibility from every operation on service objects, which is prone to programmer error. The domain model would be greatly improved.

science, software

universe of events – cosmology in software

Leave a comment

On my second reading of Three Roads to Quantum Gravity by Lee Smolin, the concept of a relational universe stands out as something fundamentally important.

Each measurement is supposed to reveal the state of the particle, frozen at some moment of time. A series of measurements is like a series of movie stills — they are all frozen moments.

The idea of a state in Newtonian physics shares with classical sculpture and painting the illusion of the frozen moment. This gives rise to the illusion that the world is composed of objects. (p.53)

In object oriented programming, the objects correspond to the particles. The focus is on capturing the state of the object, frozen at some moment of time. As methods are called on the object, changes to its state (variables) are like a series of movie stills.

Lee Smolin goes on to write:

If this were really the way the world is, then the primary description of something would be how it is, and change in it would be secondary. Change would be nothing but alterations in how something is. But relativity and quantum theory each tell us that this is not how the world is. They tell us — no, better they scream at us — that our world is a history of processes. Motion and change are primary. Nothing is, except in a very approximate and temporary sense. How something is, or what its state is, is an illusion. It may be a useful illusion for some purposes, but if we want to think fundamentally we must not lose sight of the essential fact that ‘is’ is an illusion. So to speak the language of the new physics we must learn a vocabulary in which process is more important than, and prior to, stasis. Actually, there is already available a suitable and very simple language which you will have no trouble understanding.

From this new point of view, the universe consists of a large number of events. An event may be thought of as the smallest part of a process, a smallest unit of change. But do not think of an event happening to an otherwise static object. It is just a change, no more than that.

The universe of events is a relational universe. That is, all its properties are described in terms of relationships between the events. The most important relationship that two events can have is causality. This is the same notion of causality that we found was essential to make sense of stories.

If objects are merely an illusion, and it is really causal events that are fundamental to modeling a universe that is relational and dynamical, then perhaps we should re-examine how effective object oriented programming is at producing software that effectively models real world processes. Classes of objects definitely focus on the static structure of the universe. The methods on these classes can be considered to correspond to events, which carry information in, perform some computation, and carry information out. However, the causal relationships between events is buried in the procedural code within each method; they are not expressed in a first class manner.

Personal productivity applications like spreadsheets and word processors model objects (e.g., documents) and relationships that undergo relatively simple processes involving only a few actors. The causal history of events is not as important, because there is only one set of objects in a document to maintain integrity among and the series of frozen moments model of the universe works rather well. Enterprise applications such as Enterprise Resource Planning (ERP) facilitate a multitude of parallel business processes that involve many actors and sophisticated collaborations. Each actor is performing transactions against some subset of objects, which are each progressing through a distinct life cycle. Maintaining integrity among the objects changed by these many concurrent events is incredibly complicated. It becomes important to keep a causal history of events in addition to the current state of the universe, as well as having a schedule of future events (for planning) that have not come to pass. A series of frozen moments becomes less appealing, whereas a set of processes and events seems like a better description of the universe.

software

OOPs, here comes SOA (again)

Leave a comment

Last week, I posted the following facetious comment about Service Oriented Architecture (SOA).

Posted By: Ben Eng on May 24, 2004 @ 09:27 PM in response to Message #123260

Strip all the behavior off of our domain objects. Define data structures to pass around. Provide sets of operations as stateless services, which perform functions on data, and return data to its callers. That sounds revolutionary.

All we have to do now is eliminate the object-oriented programmers, and the revolution will be complete.

There is a serious side to the issue. The recent popularity of SOA should give object oriented programmers much pause. Where do objects fit in the world of services? Are objects passé?

In the book Bitter EJB (p.57), the distinction is made between service-driven and domain-driven approaches. I am strong proponent of domain-driven approaches. However, maybe three years ago, I began feeling that a purely domain-driven approach was awkward for modeling object graphs with very sophisticated global constraints. This is a natural topic of interest, considering that I model telecommunications networks for a living.

Factoring out the global constraints, instead of embedding them within the domain objects, seemed natural. I started to develop stateless session local interfaces (using plain old Java objects, not EJB), which use domain objects as parameters and return values. Then I found myself adding conversational state to the session objects. I’m not sure, where I’m going with this yet. We’ll see after a few more weeks or months of experimentation.

After having developers fumble around with EJBs for the past four years, I am becoming disillusioned with the technology. XML and Web Services is going even further down the road of coarse-grained distributed components, which are a consequence of high latency due to expensive data marshaling and remote communications. Whenever EJB or Web Services is involved, the client code looks like garbage – calling functions on data, rather than methods on objects. Optimizing the interface to reduce client-server round trips pollutes the interface, until it also looks like garbage; objects become so polluted with denormalized data that the concepts are no longer recognizable. When faced with sophisticated object graphs, rather than simple services, the approach leads ultimately to garbage and more garbage. I’m tempted to call the approach Garbage Oriented Architecture, rather than SOA.

innovation

man-qua-universe

Leave a comment

I am currently reading Three Roads to Quantum Gravity. I have just completed the first part of the book, which is a lengthy introduction. The author felt a strong need to prepare the reader’s mind for thinking about the universe differently from whatever misguided education and personal experience was filling his head.

Perhaps if I had not already read The End of Time: The Next Revolution in Physics, I would have needed this preparation. However, being already convinced that space and time are invalid concepts for properly describing reality, I felt perfectly at home with the author’s perspective.

Something I was unprepared for was the realization that objectivist epistemology may be overly constrained by its man-qua-man perspective. Scientists, who practice the Scientific Method, impose constraints on their thinking by considering themselves an objective observer. Those who do this cannot effectively practice the science of cosmology, because no such artificial (mystical) separation can be established between observer and universe.

Only man-qua-man can allow himself to be so easily fooled into thinking the world is as he himself sees it, and be convinced that this is an accurate view of reality. Only man-qua-ordinary-man would so easily accept space, entities, and their attributes to represent the true state of things, while changes in state are viewed as snapshots in time. Most men still believe that reality is organized into a three dimensional grid representing space, and an extra dimension of time. Even their inability to sense these alleged characteristics does not bother them. Ask them to show you space and time one day, and see how far they get trying to use existents to describe non-existents.

Man-qua-ordinary-man is in fact a mere fool. In order to understand cosmology, we must become man-qua-universe and think likewise.

[It might seem strange that a software architect would spend so much effort on cosmology. Cosmology and software architecture are actually similar. Cosmology and software architecture are both about studying the universe. Just as every ordinary cosmologist has failed to gain a proper understanding of the universe, by being unable to accept that he himself cannot be separated from it as objective observer, ordinary software architects are in the same bind.]

software

Business vs Software Use Cases

Leave a comment

Let us examine how Use Cases are defined in the context of a business, who uses a software system, and a developer (vendor), who supplies that software.

Use Cases are intended to capture the software requirements. This is done by identifying and describing the units of work (Use Cases) that are performed by the users. Users are who are known as Actors. In this way, the behavior of the system is captured by analyzing how the Actors interact with the software in achieving their goals. In layman’s terms, I sometimes describe Use Case analysis as understanding the problem.

The business wants to describe Use Cases in terms that are familiar to it. Its Actors are named according to the roles within its organization. Its Use Cases are described according to the organization’s business processes. The actions and entities are named according to verbs and nouns that are specific to the business.

Meanwhile, the software developer wants to describe Use Cases in terms that can be generalized to the larger market for such software systems. Its Actors must be generalized to accommodate the many organizations that are typical within its market. Its Use Cases are described according to generalized business processes that are typical within the industry. The actions and entities are named according to abstract verbs and nouns that are not specific to the one particular deployment.

This difference in perspectives causes a communications gap. The business speaks in concretes that are specific to its organization. The developer speaks in abstractions that are generally applicable to the market. There may be a many-to-many relationship between the concretes identified by the business and the abstractions identified by the developer. The perspectives must be reconciled to allow the two parties to communicate effectively.

The abstractions provided by the developer become design elements to realize the business Use Cases.

software

mda – model driven architecture

Leave a comment

I explore Model Driven Architecture (MDA). This approach uses modeling for programming. Models drive the generation of code according to patterns.

We tend to write a lot of similar code over and over. If you have ever written code to persist a Java object into a relational database table, you know what I mean. It is similar for every object. Although the characters you type are different, there is a definite pattern, and you know deep down that an intelligent person should not have to endure such tedium day in and day out. The same goes for a lot of user interface code, like data entry forms. The same code gets written again and again; only the names are different.

I believe very strongly that model driven architecture is the way of the future for software development. Objects led to interfaces. Then, collaborations. Then, patterns and pattern languages (families of patterns). Model driven architecture is a natural extension of this trend. After identifying a pattern language, it should be possible to instantiate those patterns (write a software program) by expressing that intent through modeling. Modeling in the abstract sense is the act of capturing concepts in a notation.

I consider programming in Java to be a form of modeling. A programming language is not a very compact modeling language. Programming languages are designed to have a lot of redundancy to allow for error checking. Patterns in the code also indicate a lot of information that could be expressed in a more compact form. I view MDA as expressing software concepts in a very compact form (the model).

I think we need an example. I could write a program to represent pictures formed from geometric shapes. The software itself would probably require thousands of lines of code to perform the editing, rendering, and other functions. However, the application is really only dealing with a few basic concepts: ellipses, polygons, line segments, and a few others.

It should be possible to model each shape with a minimum number of parameters; e.g., an ellipse would be modeled by three points. In this manner, we could express any diagram understood by this application very compactly in a modeling language that captures the parameterization of shapes.

Now, let’s take that graphics package and reuse it as the foundation for constructing an application that renders telecommunications networks. This application deals with concepts like location, network elements, ports, and connections. Each concept is rendered with symbols, which are merely recurring patterns of our familiar geometric shapes.

The parameterization of shapes is no longer the most appropriate modeling language for capturing the essential concepts of this application. A more compact representation can be achieve by parameterizing the concepts that are directly understood by the application. This is the essence of model driven architecture.

Not everyone views MDA as I do. Actually, I have yet to encounter anyone who has expressed it as simply as that. Most proponents of MDA ramble on about UML and MOF and platform independence and all kinds of other non-essential mumbo jumbo. The key idea that is lost among the OMG UML/MOF folks is the notion of unit economy in the theory of concepts; that is the need to represent concepts in the most compact form. This is necessary within the mind, and this need is reflected in language.

UML makes perfect sense as a language for representing analysis and designs for software development. However, when one is looking to model concepts within any other context, the absurd verbosity and complexity exhibited by UML/MOF make as much sense as using geometric shapes as the modeling language for telecommunications networks.