Tag Archives: language

intent modeling as a programming language

In Tom Nolle’s blog article titled What NFV Needs is “Deep Orchestration”!, he identifies the need for a modernized Business Support System and Operations Support System (BSS/OSS) to improve operations efficiency and service agility by extending service-level automation uniformly downward into the network resources, including the virtual network functions and the NFV infrastructure.

Service orchestration is the automation of processes to fulfill orders for life cycle managed information and communications technology services. Traditionally, this process automation problem has been solved through business process modeling and work flow management, which includes a great deal of system integration to glue together heterogeneous software components that do not naturally work together. The focus of process modeling is on “how” to achieve the desired result, not on “what” that result is. The “what” is the intent; the content of the order captures the intent.

To achieve agility in launching services, we must be able to model services in a manner that allows a service provider to redefine the service to suit the current business need. This modeling must be done by product managers and business analysts, experts in the service provider’s business. Any involvement of software developers and system integrators will necessarily require programming at a level of abstraction that is far below the concepts that are natural to the service provider’s business. The software development life cycle is very costly and risky, because the abstractions are so mismatched with the business. When service modeling directly results in its realization in a completely automated executable runtime without involving other humans in any software development activities, this becomes Programming for Non-programmers.

The key is Going Meta. The “what” metadata is the intent modeling. The “how” metadata is the corresponding fulfillment and provisioning behavior (service-level automation). If the “what” and “how” can be designed as a language that can be expressed in modular packages, which are reusable by assembling higher level intent based on lower level components, this would provide an approach that would facilitate the service agility users are looking for. Bundling services together and utilizing lower level services as resources that support a higher level service are familiar techniques, which would be familiar to users who are designing services. When users express themselves using this language, they are in fact programming, but because the language is made up entirely of abstractions that are familiar and natural to the business, it does not feel burdensome. General purpose programming languages like Java feel burdensome, because the abstractions are for a low level computational machine, not a high level business-oriented machine for service-level automation of human intent.

Our challenge in developing a modernized BSS/OSS is to invent this language for intent modeling for services. An IETF draft titled YANG Data Models for Intent-based NEtwork MOdel attempts to define a flavor of intent modeling for network resources. An IETF draft titled Intent Common Information Model attempts to define a flavor of intent modeling that is very general, but it is far removed from any executable runtime that can implement it, because it is so imprecise (not machine executable). ETSI NFV MANO defines an approach that captures intent as descriptors for network services as network functions and their connections. However, these abstractions are not expressive enough to extend upward into the service layer, across the entire spectrum of network technologies (physical and virtualized), and into the “how” for automation, to enable the composition of resources into services and the utilization of services as resources to support higher level services that can be commercialized. More thought is needed to design a good language for this purpose and a virtual machine that is capable of executing the code that is produced from it.

going meta

Anatomy of an n-tier application

A fully functioning web app involves several layers of software, each with its own technology, patterns, and techniques.

At the bottom of the stack is the database. A schema defines the data structures for storage. A query language is used to operate on the data. Regardless whether the database is relational, object-relational, NoSQL, or some other type, the programming paradigm at the database tier is distinctly different than and quite foreign from the layers above.

Above the database is the middle tier or application server. This is where the server-side business logic, APIs, and Web components reside.

There is usually a set of persistent entities, which provide an object abstraction of the database schema. The database query language (e.g., SQL) may be abstracted into an object query language (e.g., JPQL) for convenience. The majority of CRUD (create, read, update, delete) operations can be done naturally in the programming language without needing to formulate statements in the database query language. This provides a persistent representation of the model of the application.

Above the persistent entities is a layer of domain services. The transactional behavior of the business logic resides in this layer. This provides the API (local) that encapsulates the essence of the application functions.

The domain services are usually exposed as SOAP or RESTful services to remote clients for access from Web browsers and for machine-to-machine integration. This would necessitate that JSON and/or XML representations be derived from the persistent entities (i.e., using JAXB). This provides a serialized representation of the model of the application.

We finally come to the presentation layer, which is divided into server-side components residing in the application server and client-side components that execute in the Web browser. Usually there is a presentation-oriented representation called a view-model, which matches the information rendered on views or input on forms. The view and controls are constructed from HTML, CSS, and JavaScript. The programming paradigm in these technologies is distinctly different than the layers below.

Extending the application

Let’s examine what it would take to extend an application with a simple type (e.g., string) property on an entity. The database schema would need to be altered. A persistent entity would need a field, getter and setter methods, and a binding between the field and a column in the database schema. The property may be involved in the logic of the domain services. Next, the JSON and XML binding objects would need to be augmented with the property, and logic would be added to transform between these objects and the persistent entities used by the domain services. At the presentation layer, the view-model would be augmented with the property to expose it to the views. Various views to show an entity’s details and search results would likewise be enhanced to render the property. For editing and searching, a field would need to be added on forms with corresponding validation of any constraints associated with that property and on-submit transaction handling.

That is an awful lot of repetitive work at every layer. There are many technologies and skill sets involved. Much of the work is trivial and tedious. The entire process is far from efficient. It is worse if there is division of labor among multiple developers who require coordination.

A better platform

When confronted with coordinating many concomitant coding activities to accomplish a single well-defined goal, it is natural for an engineer to solve the more general problem rather than doing tedious work repeatedly. The solution is to “go meta”; instead of programming inefficiently, develop a better language to program in. Programming has evolved from machine language to assembly language for humans to express instructions more intuitively. Assembly evolved to structured languages with a long history of advances in control and data flow. Programming languages have evolved in conjunction with virtualization of the machine (i.e., bytecode) to provide better abstractions of software and hardware capabilities. In the spirit of Guy L. Steele’s Growing a Language talk from OOPSLA ’98, components, libraries, and frameworks have been developed using a programming language that itself supports extending the language itself within limits. All of these innovations continually raise the level of abstraction to increase human productivity.

We are hitting the limits of what can be expressed efficiently in today’s languages. We have a database storage abstraction that is separate from server-side application logic, which is itself separate from client-side (Web browser) presentation. There is growing support for database and server-side abstractions to scale beyond the confines of individual machines. Clustering enables a software to take advantage of multiple machines to distribute load and provide redundancy in case of failure. However, our abstractions seem to stop at the boundaries between database storage, server-side application logic, and client-side presentation. Hence, we have awkward impedance mismatches when integrating top-to-bottom. We also have impedance mismatches when integrating together heterogeneous application components or services, as RESTful and SOAP Web Services technologies cross the boundaries between distributed software components, but this style of control and data flow (remote procedure calls) is entirely foreign to the programming language. That is why we must perform inconvenient translations between persistent entities and their bindings to various serialized representations (JSON, XML).

It seems natural that these pain points will be relieved by again raising the level of abstraction so that these inefficiencies will be eliminated. Ease of human expression will better enable programming for non-programmers. We are trying to shape the world so that humans and machines can work together harmoniously. Having languages that facilitate effective communication is a big part of that. To get this right, we need to go meta.

innovation

innovation [noun] – context violating exaptation.

Ever since I read this tweet in 2012 by Fast Company, I have redefined innovation in this way.

Here is the first definition of exaptation from Dictionary.com.

noun, Biology
1. a process in which a feature acquires a function that was not acquired through natural selection.

By taking something or a combination of things and applying it to a purpose to which it was not intended (violating its original context), one may discover that it is well-suited to perform a different function. This discovery becomes an innovation.

the literal, the colloquial, and the peace of nothing

I try very hard to write using phrases that convey literal meaning. I look down on writers who rely too much on colloquial phrases. I consider it sloppy, especially when the writer is unfamiliar with the concrete situation from which the phrase originated. Then, the phrase is merely repeated to convey a shallow meaning without understanding its deeper meaning and context.

Today, I was curious about the origin of the phrase “extend an olive branch”. I thought there must be a great, gripping story behind it. It turns out the origin has long been forgotten. It was that memorable and significant. The symbol of peace that is so widely understood across many cultures is anchored by… nothing. It is learned by rote and repeated mindlessly, not knowing that it is hollow and empty. Perhaps that becomes the truly significant origin of the modern phrase. Contemplate what new and deeper meaning this would assign to the phrase “extend an olive branch”, when applied to present day situations like the Middle East peace process. There is a certain perfection to that. Shaka, when the walls fell.

war of words

If the pen is a weapon, words are the ammunition. Politics is marketing, and marketing is a war of words. In both realms, the goal is to sway public opinion with ideas. A good tactic for undermining the opponent’s position is to poison the language they use to identify themselves.

For decades, whenever a leader wishes to be resolute about a cause, they declare a war as a call to arms. We have seen the war on drugs, war on crime, war on poverty, war on AIDS, war on terror, and now even a war on obesity. The indiscriminate use of war parallels the prosecution of indiscriminate wars in Bosnia, Iraq, and Libya. This enables the term “war on” to become vulnerable to poisoning, so that future use of this term becomes toxic. Furthermore, “war on” is conveniently similar to “moron”, which makes for a perfect nexus of ideas (superficial and with substance) for contaminating the term “war on”. We can now apply “war on” as a derogatory adjective; for example, should the President declare another war on something, he can properly be identified as yet another war-on President. War-on decisions become acts of foolishness and symbols of an ignorance of unintended consequences.

programming for non-programmers

Users want to express their desires to a computer and have the computer fulfill their wishes on demand. We frequently see this desire manifested as software “requirements” for configurability or customization without the need for programming. Users view programming as a highly technical, error prone, and cumbersome chore that should be left to professional software developers.

Users would like developers to produce software that is capable of intelligently adapting its structure and behavior to the needs of the day, as the environment and the business requirements change. Users believe they should be able to inform the software of these requirements through intuitive visual techniques. Click on a tool and fill in some information. Drag and drop some icons across the screen. New computations, new algorithms, and new ways of doing business should be configurable with a few clicks by users, who know nearly nothing about how a computer executes instructions, the nature of such instructions, and how such instructions come into existence.

What users do not realize is that the act of expressing themselves through clicks and gestures to produce instructions for the computer is itself an act of programming. The complexity involved in formulating those instructions is proportional to the degree of intelligence intrinsic to the computation. The breadth of requirements that can be satisfied by those instructions is a function of how much can be expressed by the user. Narrow coverage of requirements implies an inflexible system that targets a niche problem domain. Broad coverage of requirements implies a general purpose system.

General purpose programming languages have traditionally been expressed as text. Like all languages, programming languages have a vocabulary, words formed from a character set, and a grammar, which governs how to organize those words in meaningful ways. Textual programming languages allow humans to express themselves concisely in a form that is both precise enough for a computer to comprehend and similar enough to natural language (e.g., English) to be intuitive. General purpose programming languages have naturally evolved from primitive instructions close to the machine (e.g., assembly language) to abstractions that correspond to natural language. Text is a highly efficient and flexible method of communicating ideas. An alternative technique using graphical images may be applicable for expressing ideas that can be tangibly visualized, but images are wholly inappropriate for non-visual ideas. An image may be able to convey a thousand words, but to have control over exactly what words to formulate would not be easy by using visual techniques.

We can expect that programming languages will continue to advance to become more intuitive to humans and more efficient at expressing complex instructions to machines. We can also expect that graphical techniques will continue to evolve to make programming easier to both professional programmers and users, who don’t want to program. What we must accept is the fact that instructing a machine is programming, regardless of whether it is expressed as text or using visual techniques.

it’s fun to shoot people

Regarding today’s current events surrounding Marine general: It’s ‘fun to shoot people’:

Many people are trying to defuse the situation by saying that Lt. Gen. James Mattis’ statements were a “poor choice of words” or “flippant”. They are spinning it, because they want to portray Mattis’ statement as either not reflecting the man’s (or the Marine Corp’s) convictions or not intellectually or morally significant. They are conceding the superficial impression that Mattis’ statements are morally reprehensible, which can occur when the reader deliberately interprets the words out of context.

The spirit and true meaning behind Mattis’ statements were naked and explicit. His words were unsophisticated but not a poor choice. His choice of words should make a rationally moral man proud. However, for those who are offended by Mattis’ “poor choice of words” or “flippant” remarks, allow me to restate using words that bring clarity to the moral basis underlying Mattis’ intent.

Afghanistan has been plagued by those, who believe in oppressive practices, such as violently punishing women for such ridiculous transgressions as being seen in public not wearing a veil. I have a strong moral conviction to condemn those who support such inhuman practices. As a proud member of the Marine Corps, I can speak for all men who uphold justice as a moral principle, when I say, it is our honor and pleasure to be the agents for delivering just consequences in open combat on the battlefield upon the forces of evil. Those who believe that brutality and force are tools for proper human behavior are now learning, as they deserve, the logical consequences of living according to — and dying by — the principles, in which they so strongly believe.

paleo-mock-semantology

New Scientist published an article titled Dinosaurs’ ‘bulletproof’ armour revealed.

Apparently the massive herbivore, ankylosaur, developed armor plating for protection from its enemies. This adaptation was necessary during the late Jurassic and Cretaceous periods, because rival species began deploying rudimentary ballistic weapons. Gun violence became a widespread social problem among dinosaur clans. Fossil evidence has revealed that, as predatory practices became more sophisticated, depleted uranium penetrators and shaped charge warheads rendered the ankylosaur’s defensive systems combat ineffective. It is believed that rapid advances in delivery systems and energy yield ultimately led to the mass extinction of ankylosaurs (and many other species), when a multi-gigaton payload was launched from long range, detonating near the Yucatan Peninsula.

language design 101

Language provides us with the framework with which we express ideas. Numbers, arithmetic, and calculus are the language of mathematics. C, C++, and Java are languages for computer programming. This is only partly true, as Guy Steele Jr. explains in Growing a Language. This perspective on language design extends far beyond programming.

I will focus on software related language design. It is an exercise for the reader to extrapolate how the same principles apply to literature, music, and other forms of human expression.

Just as Java is the most basic vocabulary and grammar with which more expressive languages can be designed, English is similar. The language required to produce a book is far more advanced than English vocabulary and grammar. Literary devices, plot structure, and character development are patterns that are designed into the language of literature. Similarly, design patterns make Java a useful language for programming.

The language of object-oriented analysis and design (OOAD) is the Unified Modeling Language (UML). However, UML is not very expressive by itself. It was not designed to be as precise as programming languages, because it was motivated by the needs of conceptual modeling. They thought that conceptual modeling is not precise. That is the tragedy of imprecise thinkers.

Now, they have caught on to the idea of using models to generate code. This requires precision; a machine only understands 1 and 0. There is no room for ambiguity. It is now a huge challenge to apply UML to this model-driven architecture (MDA).

We are constantly grasping for ways to improve how we express ideas. A thousand words are better expressed with a picture. The concepts represented by the geometric shapes are better expressed with tags. Patterns are recognized as we organize those concepts into more advanced ones. As a consequence, many thousands of instructions are captured in a blueprint with a small number of symbols. At each step, we seek to be more expressive with compact representations. This is language design.

communication and notations

Most of my life is spent communicating, despite the fact that I rarely say much. Ever since I read about epistemology and the theory of cognition, I’ve recognized the need to build conceptual models first and notations second. I continually yearn for a precise method of expressing ideas.

All around me, I see people misunderstanding each other. Communication is a shared responsibility. It requires a collaboration between the source and the target.

The source is responsible for organizing his conceptual model. He identifies the distinctive subset of the model that requires expression, and separates it from the remainder of the model that is the foundational context. He structures a presentation for the expression using a notation.

The target is responsible for comprehending the notation. He must decode the notation into the foundational context and distinctive subset of the model that is being expressed. Ideally he decodes exactly the same conceptual model that originated from the source. However, more often than not, the resulting model is different. Communication has failed to faithfully convey the meaning.

Modeling is my life’s work. I do it well. Unfortunately, I am unskilled in developing notations. I must settle on using the notations that others have handed down. For now, I will be misunderstood. And so will you.