Week 29/30/31 of 2010

I've been a bit busy over the last few weeks and didn't manage to slot in any blogging time. Sorry about that. The majority of my effort over that time has focused on a redesign of the attribute hierarchy and a reimplementation of various attributes in Attribute. There were a number of problems and limitations with the previous design which have been bugging me for some time now.

As a consequence, I completely rewrote the Attribute Classification technical note. Normally I would publish all technical notes on this blog first while also publishing another version (which I update over time) on the www.ooatool.com website. However, since I have started making the website versions notation-switchable, the blogged version has become less and less useful. Especially since the majority of current users switch to Executable UML notation anyway (and I still prefer Shlaer-Mellor notation). Thus, I will no longer publish technical notes in full here. I've also dropped any reference to creation date from new technical notes since the last updated date is much more relevant than when it was created. I recommend long-standing users scan the above technical note switching between notations since there are quite a few changes between the notations. The reworked Information Model subsystem of the OOA of OOA is shown below:

A reworked Data Type subsystem is also shown below:

With data type usage extracted into a new Data Type Usage subsystem:

During the redesign of the attribute hierarchy, I made a number of significant changes:

• Literal attributes are distinguished separately from simple attributes now. As a consequence, default and initial values no longer have final statuses. I've also changed the attribute suffix in Shlaer-Mellor from "F" to "L" while Executable UML still uses "frozen" and Executable UML2 still uses "readOnly".
• Users will now only be able to specify data types and conditional statuses for base attributes (previously, manual data types and conditional statuses could be set on all attributes). As a consequence, data types can't be incompatible anymore (except for polymorphic base attributes which are treated specially).
• Conditional statuses of referential and polymorphic attributes are now assumed to be false unless proved to be true (previously, we assumed true unless proved to be false). This reflects the idea that conditional attributes are the exception rather than the norm.
• The qualifier polymorphic is now mapped to abstract in Executable UML and we no longer use "P" suffixes to identify such attributes. Instead we use an "abstract" suffix in the browser and omit the suffix entirely on class diagrams relying on the use of an italic font.

I've also been working on a design for UML aggregations and compositions which I will discuss in full after it's implementation is complete (and I can show some class diagrams with diamonds on them). These concepts deal with ownership across relationships and I have been calling them child-parent relationships in Shlaer-Mellor so far. However, I'm planning on eliminating the terms child and parent. I'm also planning on renaming Parent Allocated ID Type to Owner Allocated ID Type (or Aggregate Allocated ID Type in Executable UML).

Week 28 of 2010

I've been very busy over the last week, extending and evolving the OOA of OOA. I've also been implementing some OOA Tool changes with regard to Arbitrary ID Types which have been completely overhauled. Let's start with the Recursive Design subsystem which is shown below:

This hasn't changed much over the last week. The Location attribute on Software Product has expanded into several attributes reflecting the new implementation within OOA Tool. The user can set a manual location (relative or absolute). However, a default location (relative if parent location defined, absolute otherwise) is always defined and used if no manual location is specified. A parent location (absolute if defined) is always defined for software products enclosed within other software products, e.g. code modules are enclosed within projects and/or external projects. Manual and default locations are resolved relative to any parent location. All locations are specified as URIs while the final location should also be a valid absolute URL. Using URIs here blurs the distinction between file system and web locations. OOA Tool also converts to and from platform specific paths allowing normal file chooser dialogs to be used.

Another change involves the new Work Product object, a subtype of software product. This change expanded so much over the week that a new Work Product subsystem had to be added to the OOA of OOA. This sounds like a major pollution of the domain, especially since I have previously mentioned that I wanted to evolve the Diagramming domain used in the OOA Tool project into a more general Work Product domain covering diagrams, tables and other graphical constructs. However, I'm only concerned with OOA/RD specific information in the new subsystem. No mention of size, location or other graphical pretty printing is involved. The Work Product subsystem is shown below:

As this subsystem evolved from a few objects into 25 objects, I made a number of important discoveries:

• each diagram product has it's own notation which may differ from the enclosing project's notation reflecting the reality in OOA Tool while table products do not,
• a new Information Model Partition object was required to formalize this subsystem and as a result, I was able to simplify the messy unassigned/assigned object/relationship cluster in the Information Model subsystem,
• archetypes can determine diagram locations (e.g. when generating reports) from a metamodel population by looking at instance data associated with this subsystem,
• imported objects and event destinations are modelling issues best captured in the OOA of OOA rather than as diagram pretty printing,
• tasks and task activities (now called Work Cells) have a proper home and it's in this subsystem,
• and I identified the need for allowing generalized data types in polymorphic attributes since I can't resolve the polymorphic row and column attributes on Table Cell without it (more soon).

The new subsystem also makes bridging to a generic Work Product domain much easier since each diagram and table product now has an object in the OOA of OOA domain which can be mapped to a counterpart in the generic domain. The new Information Model subsystem is shown below:

You should see how the new Information Model Partition object eliminates the need for unassigned/assigned object/relationship objects. I haven't made the change in OOA Tool yet since it will probably involve lots of work. However, the above subsystem is much cleaner with the change.

You may also notice that I have replaced Arbitrary ID Attribute with System Set Attribute. System set attributes allow system reallocated IDs, current subtype and current state attributes to be added to an object (and provides room for future system derived attributes). As far as current state is concerned, OOA91 allowed users to access it while OOA96 disallowed such access. I'm in favour of allowing controlled access to current state information. My initial plan was to provide a special current state (and current subtype) expression syntax within the Action Language to support this. However, this is messy since it complicates the Action Language and hides such access from any model reviews. The better approach is to allow users to add a current state attribute (which is set by the system) to an appropriately related object, e.g. the current state of a lifecycle model can be added to it's active object. I already automatically define an Enumerated State Type for each state model allowing such an attribute to be typed. I've extended this idea to allow the current subtype of a subtype-supertype relationship to also be added to an object since I also already automatically define an Enumerated Subtype Type for each subtype-supertype relationship. This allows identifiers to be defined which include the current subtype! Current subtype attributes should only be added to the appropriate supertype object. As a result of the overall change, I've also replaced the "(A)" suffix with an "(S)" suffix within information models.

I was also going to talk about the revamped Arbitrary ID Type which I've already coded up but time has run out (I'm trying to limit weekly reporting to a few hours a week now!). I'm definitely planning a new OOA Tool build now since the current metamodel won't load into the currently released build.

Week 27 of 2010

I've had a very productive week modelling wise. I've been trying to finish off the Recursive Design subsystem in particular. However, I've also been resolving a few cross-subsystem modelling issues as well. If some on you are wondering what's happening with the OOA Tool 2 development. It relies heavily on the OOA of OOA unlike OOA Tool which quietly added a few extra details not in the old OOA of OOA, e.g. location information and code module support. I also want to ensure populations (including runtime populations) are fully defined in the OOA of OOA before coding the virtual machine in OOA Tool 2.

I realised one nasty issue with regards to metamodel population objects was caused by an identifier simplification that I've been using for a long time now. The simplification in question is the use of a standalone arbitrary ID attribute for identifying information models when I should have used the enclosing domain's compound identifier. The advantage gained was being able to refer to an information model using only a single referential attribute. The disadvantage was that several objects need to refer to both an information model and the enclosing domain using different referential attributes. I could have added loop constraints to ensure the referenced information model is the one defined by the referenced domain. However, I haven't being adding those constraints to date. The nasty issue only surfaced during a moment of reflection - metamodel population objects actually need to refer to two different projects, the first being the population project and the second being the metamodel project. The extent of this dual project relationship is something that definitely should be visible in the OOA of OOA but was being hidden by my identifier simplification. I've now entirely eliminated the arbitrary ID defined in information model using the enclosing domain's compound identifier instead. This took many hours! The moral of this story is don't shortcut compound identifiers in complex domains.

Now back to the Recursive Design subsystem, the latest version of which is shown below:

I've now added support for Code Modules since most developers want to bundle code into manageable files in the real world. Operations can still be coded within the model but the user will now be able to locate the actual code inside an external Action Module associated with the enclosing domain. I will have to extend the Action Language syntax to include operation definition statements so that multiple operations can be defined within a single code module. However, this change will make code reviews and configuration management much easier. Code modules also include archetype templates (see Archetype Language) and pattern sets defining syntax and lexical grammars (see Pattern Language). This does offer the possibility of manageably supporting multiple languages since we could associate a particular language with a given code module.

I've also added the concept of a Software Product which is a locatable software component at the highest level. I haven't used the word component since it would clash badly with the UML component concept. Software products include code modules (discussed above), projects and external projects (loaded from ".ooa" files), and output products generated from translations. The current version of OOA Tool also implements a few custom output products, i.e. diagrams and information model reports. Information model reports will be reimplemented using archetype templates in the future. However, diagram image generation will have to remain a custom output product for now since I imagine creating a ".png" template directly is a really difficult task!

All software products have a lifecycle. Code module lifecycles include states such as loaded, parsed, changed and saved. Project lifecycles include states such as changed and saved. External project lifecycles include states such as loaded. Output product lifecycles include states such as generated, saved and loaded. A related active object is External Asset which defines states such as loaded (has a similar state model to external project). Obviously, all of the mentioned state models will also define one or more error states etc. I haven't finished modelling the various state models yet so I can't include them here. I originally tried to keep this concept out of the OOA of OOA. However, that was a mistake with regards to the Recursive Design subsystem since it's purpose is to define how domains are integrated and reuse enabled. On the other hand, I don't like having the Task and Task Activity objects in the OOA of OOA which is almost certainly domain pollution. However, I'll keep them for now since I want to maintain support for Project Matrices (one of the original Shlaer-Mellor work products) without adding a new project management service domain. I did move (a while ago) this pollution from the Recursive Design subsystem into a separate Recursive Design Project Management subsystem to isolate it.

I've also added asset dependencies to the model. Domains should have no dependencies since they define a world in isolation. Bridges and layers obviously have domain dependencies. Input populations also have domain dependencies. Transformations have domain and input population dependencies. They may also have simulation dependencies if part of an input population is generated. There should be no code module dependencies across assets since operations remain hidden within their enclosing domain. The same should be true for pattern modules. Even though syntax might be shared across domains, e.g. I've reused some bits of Java syntax within the action, archetype and pattern languages. I'm not planning on formalizing any such sharing. Archetype modules are a little bit different since I could imagine wanting to share some templates across multiple software architecture domains. This is why I have captured the includes relationship on the model. However, changing a shared template could completely break one or more of the software architectures unless very careful consideration was given to any changes. Balancing the risks here, I'm currently going to require any included archetype modules to be defined within the same domain as the includer. This places the responsibility of checking any changes on the user which I believe is required in this situation. Comments welcome.

Remember, we are enabling Asset reuse, not Code Module reuse in the OOA of OOA. Code is simply implementation logic within an asset.

I was also going to talk about a bunch of other stuff today but it's already 7pm and I want to do some real work tomorrow! I'm almost certainly going to release a new build of OOA Tool now before OOA Tool 2 is rolled out since I'm making lots of changes to the OOA of OOA some of which I want to use to model the OOA of OOA! Anyway, enough for now.

Week 26 Addendum

I finished the report yesterday and then headed off to the coffee shop to reflect (and flesh out the next modelling task). In particular, I thought about how I might access thread pool allocation colouring within an archetype template to generate the necessary code. It occurred to me that I would need to manually create a new colour for each thread pool instance using yesterday's model - not good! After some rework, I ended up with the following revised Colour subsystem:

The above model adds the concept of an Indexed Colour (I did call it an Array Colour for a while) which allows a colour to be defined which implicitly defines a component colour for each indexed element. Object instances are used for indexing purposes. Using the thread pool allocation issue as an example, "Thread Pool Allocation" would be an indexed colour indexed by instances of Thread Pool. Each instance of thread pool that a user creates within a simulation results in an implicit component colour of "Thread Pool Allocation" being created. To allocate event destinations etc. to a given thread pool, the user creates an Indexed Colouring Rule which references the "Thread Pool Allocation" colour and the given thread pool's object instance (along with the contents of the rule of course!).

We also need to add support for indexed colours to the Boolean binary operator which tests for colouring in the Archetype Language:

objectInstance ( coloured | marked )
    [ domain '@' ] colour [ '[' objectInstance ']' ]

Where the first objectInstance expression references the instance being checked and the second references an instance of an architectural concept (e.g. thread pool). To demonstrate the above, yesterday's example archetype code fragment is expanded below to support a PersistanceControl object from a user defined Persistence domain:

.Select many objects from instances of Object
    where selected coloured Persistence@Persist
.for all object in objects
.Select one persistenceControl
    from instances of Persistence@PersistenceControl
    where object coloured Persistence@Persist[selected]
.// Generate code to persist using control settings...
.end for

From the above, we can see that indexed colours can be checked for in a generic sense, i.e. all objects with any "Persist" colouring are selected at the start.

The only other change I made to the Colour subsystem was to refactor two ownership relationships into one. Specifically, top-level colouring rules are defined by either input populations or transformations. Since both are assets, I added a new Colouring Rule Owner object between them and asset. I'm now able to use a single defines relationship for top-level colouring rules.

Week 26 of 2010

I've continued work on the Recursive Design, Population and Colour subsystems of the OOA of OOA. I dropped the term Colour Model as it didn't add any value. The Colour subsystem is shown below:

The basic points to note are:

• Any domain can define logical Colours (called Marks in Executable UML), not just architectural domains since any domain can define archetypes for translation (e.g. a non-architectural domain may create an archetype to generate an external configuration file and use one or more colours to control the process). Even the OOA of OOA domain defines a number of colours (see next bullet).
• Colouring Rules (called Marking Rules in Executable UML) are defined by either input populations (allowing boundaries to be defined), simulations (allowing thread pool allocations, breakpoints and logging to be defined) or translations (allowing archetype specific colouring to be defined). All colours used with input populations and simulations are predefined by the OOA of OOA domain.
• The result of applying a colour rule prior to a translation is always zero or more Coloured Object Instances (called Marked Objects in Executable UML) referencing either metamodel or model object instances. The fact that we treat metamodel data just like ordinary model data makes the process here very simple.
• Visual colours are outside the scope of the OOA of OOA. Instead they are associated with work products such as diagrams and tables. The idea being that on a given diagram (e.g. an OIM) a user can associate a particular visual colour (e.g. green) with a colouring rule (e.g. the boundary rule associated with a specific input population) which applies a specific logical colour (e.g. Boundary defined by the OOA of OOA domain) to it's data. Such colour selections are normally transient.

The need for input population boundaries is particularly strong where generated populations are concerned. Metamodel populations and output populations from long simulations can be really big. If your transformation only uses a small subset of such data then a boundary can be defined by colouring objects within the boundary. Population data outside a boundary is never generated as long as the boundary is specified on the generated population itself, not a composed population which contains the generated population. One may even want to specify a boundary on a simple population to stop users adding specification data outside the scope of the population's intended use.

The need for simulation colouring rules is weaker since colours can't be accessed at runtime without providing access to metamodel data. However, the mechanism does allow concepts such as thread allocation, breakpoints and logging to be easily specified. Other concepts may be added in future. One benefit of using colouring rules here is that a translation can access them quite easily (e.g. to add simulation specific logging to generated software).

Population Selection Rules are defined using a series of inclusion/exclusion criteria. The first rule will always be an include rule. Any children will be exclude rules. Any grandchildren will be include rules etc. For example, the top level include rule may select a particular domain, with child rules excluding particular subsystems. We can combine these rules with Set Expression Rules for union, intersection and difference. Many top-level colouring rules will actually be a union set expression rule containing multiple include population selection rules. Thus, we probably don't need population selection rules to be any more flexible. A possible exception concerns the Where Selection Rule which only allows limited expressions to be defined at present.

What's not obvious from the above model is how colours are used to control archetype translations. This is done by adding a new Boolean binary operator to the Archetype Language:

objectInstance coloured [ domain '@' ] colour

Where objectInstance is an arbitrary object instance expression and domain is the name of the domain which defines the statically referenced colour (which is optional if colour defined in archetype's domain). The operator simply checks for the presence of a coloured object instance for the specified object instance. An Executable UML version of the operator will also be supported:

object marked [ domain '@' ] mark

Colours are always statically referenced within archetypes allowing OOA Tool to track colour usage and rename colours automatically if necessary. I can't see a real need for archetypes to dynamically access colours at present. An example archetype code fragment is given below:

.Select many objects from instances of Object
    where selected coloured Persist
.for all object in objects
.// Generate code to persist...
.end for

Once again for those Executable UML users out there:

.Select many classes from instances of Class
    where selected marked Persist
.for all class in classes
.// Generate code to persist...
.end for

I won't go into anything else this week since I've probably already said enough! If anyone has any views on how I'm intending to support colours (and marks) then please comment (or drop me an email).

Week 25 of 2010

This week's report is a little late since I thought I might finish some modelling first. The only problem with that is that modelling often leads to more modelling! I've been working on the Recursive Design and Simulation subsystems of the OOA of OOA. In fact, I've now replaced the old Simulation and Translation subsystems with new Population, Population Runtime and Colour Model subsystems. Most of the old Simulation subsystem was actually concerned with population data anyway. I've moved the few objects directly relating to simulation and translation into the Recursive Design subsystem which is a good home for them anyway. The new Recursive Design subsystem is shown below:

You can now see that recursive design involves partitioning a system into domains, bridging between domains, adding layers to those domains if appropriate (new in OOA10), populating those domains, simulating the system for testing and translating the system into the final software product. Recursive design also involves colouring arbitrary aspects of the model and/or data to control aspects of simulation/translation such as population boundaries and thread allocations. I'm providing support for basic thread pools during simulation to help facilitate successful reuse, i.e. users should always test concurrent models using multiple thread simulations. There are many other architectural concerns that are not supported during simulation. Those concerns can only be tested by running a translated version of the system using an appropriate software architecture. I may need to add support for additional architectural concerns in future, e.g. concurrent access controls. Any comments here?

Colour models and Colours in Shlaer-Mellor are known as Marking Models and Marks in Executable UML. This is a neglected area of Shlaer-Mellor methodology and not well defined in MDA either. However, the need for those ideas became evident as I started to think how I would capture thread allocations for simulation purposes and population boundaries where I want to include a subset of a given population in a simulation/translation (a good example here are metamodel populations which rarely need to used entirely). I'm still working on the Colour Model subsystem so I won't show it here. I have been thinking how I can incorporate colour models into OOA Tool but I'll leave that topic to another day also. One interesting issue with colours is how you access that information within archetype templates which traditionally only accesses object, attribute and relationship instance data.

Back to populations. The new Population subsystem is shown below:

I've made quite a few changes here. Populations are now either Input Populations which are project assets in their own right (i.e. can be imported into other projects), Executable Populations used with simulations or Translatable Populations used with translations. Input populations are Simple Populations (i.e. user data), Metamodel Populations, Composed Populations allowing a set of input populations to be merged into one, and Output Populations which are derived from the output state of a completed simulation run (e.g. a software architecture may code an optimizer as a model that can be simulated to generate optimization flags that can then be passed as input into a translation). Composed populations is a powerful concept when combined with output populations allowing multi-stage build processes to be defined.

An interesting issue with translation is what do you do about derived attributes and relationships? Do you use an active population during translation so that derived attributes and relationships are still determined on demand or do you pre-calculate all such information before you start the translation. I've taken the view that pre-calculation is best so that translation will in the most part be a deterministic process, i.e. if you recalculate on demand then the results may change if the calculation logic is non-deterministic (e.g. involves current time). However, this means that I can't set any attributes during translation (which is probably a bad idea anyway). In the above model, a translatable population is an input population with all derived data pre-calculated.

Population data and executable population data is shown in the model above. However, runtime population data could not also be shown without considerable mess. Thus, I've put all supplemental runtime population data into a new Population Runtime subsystem. This was the work I was trying to finish before I did this report but I haven't quite finished it yet so I won't include it here yet. An additional requirement here is that sufficient data be gathered so that an automatically generated Thread of Control Chart (or Sequence Diagram) can be associated with each simulation run.

So I plan on finishing the Population Runtime and Colour Model subsystems now. I'm considering whether it's a good idea to add support for what I've done here to OOA Tool immediately without waiting for OOA Tool 2 to be completed. It would allow me to validate some of the ideas. The problem is that without a working Action Language and Archetype Language, it doesn't give us much.

Week 24 of 2010

I worked on the Simulation subsystem of the OOA of OOA most of the week since OOA Tool 2 needs an OOA virtual machine immediately (all derived calculations will be coded using Action Language code rather than Java code). The bulk of the Simulation subsystem focuses on data populations and the instance data associated with different types of population.

Simple populations define Object Instances, Base Attribute Instances (simple, arbitrary ID and mathematically dependent attributes), Relationship Instances (simple, associative and subtype-supertype relationships) and Participant Instances. It also defines Counterpart Mapping Instances allowing object instances in one domain to be converted to counterpart object instances in another domain within bridging code. The only population dependent values that can be used in simple populations are object instance values. Any attribute instances with a population dependent type that isn't an object instance type will have an undefined value.

Arbitrary ID attributes have a current value representing an allocated arbitrary or ordinal ID. The system automatically allocates or reallocates IDs as and when needed within limits controlled by the analyst, e.g. ordinal IDs reflect an ordering which must be preserved. Within limits, these IDs may be used outside of the system. For both of these reasons, arbitrary ID attributes require instance data. I really must finish writing up the technical note on arbitrary IDs in OOA10!

Mathematically dependent attributes can be calculated in many different ways. The simplest is on demand which doesn't require instance data. However, a minimum duration between recalculations can also be used by the analyst to restrict how often resource intensive calculations are performed. Presentation data rarely needs to be calculated more than once per second. We may also want to write calculation code which uses previous values to optimize the current iteration (verifying that the previous value is still current may be very fast and worth the effort). These kinds of analysis optimization are difficult to automate at present but should be used sparingly. None of these reasons require a current value to be persisted long-term. However, they do present a need for instance data at some point.

Referential and/or polymorphic attributes have no instance data since they are calculated on demand which generally isn't often since their main purpose in life is to ensure the model is formalized. Composed and mathematically dependent relationships also have no instance data since they are also calculated on demand.

Executable populations extend simple populations with Event Destination Instances, Event Instances, Event Data Item Instances, Operation Instances and Input Parameter Instances. It also defines Threads and Thread Allocations allowing concurrent processing resources to be allocated. The population dependent values that can be used in executable populations are object, event or operation instance values. Return coordinate and transfer vector values can't be used.

Simulations are started by generating an event or invoking an operation. An executable population allows us to define a set of triggers which can be run. It also allows us to define a thread allocation policy which is the minimum information required to execute a realistic concurrent simulation. A default mapping to multiple threads for concurrent simulation purposes is not useful in my view. If you aren't interested in defining a thread allocation policy then use a single thread. I should point out that some systems may require a minimum number of threads to avoid deadlock, e.g. invoking a synchronous service across a bridge will generally block a thread.

Runtime populations extend executable populations with Generated Event Instances (specifying whether delayed and/or recurring), Invoked Operation Instances, Output Parameter Instances and Return Communication Instances (allowing us to track return coordinate and transfer vector returns).

Generated event instances are queued on threads as determined by the thread allocation policy and are consumed as determined by OOA policy. Delayed events block the stream of events between a source/target pair of event destinations (key OOA policy is to ensure sequential processing of events between source/target pair). Recurring events are re-queued after they are consumed allowing heartbeats to be implemented. However, re-queued events may not stop a simulation from completing.

Invoked operation instances are logically pushed on to threads as invoked and pulled off when invocation completes. However, these instances may persist for various reasons, e.g. logging. A thread will also block after invoking a bridging operation mapped to a synchronous service awaiting a synchronous return operation with the correct return coordinate. A thread can also block for other reasons, e.g. awaiting input/output.

Return communication instances allow us to track how timely and often returns occur. Return coordinates must have a single timely return since a thread is waiting for the results. If a timely return does not occur then the system may generate a default return and any later return would then be flagged as an error. Transfer vectors are more flexible as determined by the analyst. They may be conditional or have a multiplicity of many. However, if not conditional then a default initial return may be generated if a time constraint has been specified. Whether any later return is flagged as an error would then depend on whether multiple returns are allowed.

Once I finish tidying up the Simulation subsystem then I will update the OIM on the website. I also need to ensure I have a clean workable design for representing runtime models in OOA Tool 2 based on this work. I also want to be able to generate Thread of Control Charts (and Sequence Diagrams for Executable UML) from runtime populations on completion.