Week 12 of 2010

Work is progressing nicely on OOA Tool II. I spent some time getting the new GUI framework started this week. I've been able to drop some nasty workarounds from OOA Tool I that were needed because of J2SE 5.0 bugs. There are a lot less bugs in Java SE 6.

I'd like to talk about a mapping from objects, attributes and subtype-supertype relationships (SSRs) to tables and rows. Within OOA Tool I, object instances stored as rows were mapped directly to objects. However, the design required a knowledge of subtype-supertype relationships and an ability to migrate between subtypes while preserving object instance references. I wanted to take SSRs out of the picture entirely in the more generic OOA Tool II architecture. Otherwise, I would need to add the concept of SSRs into the Data Management dictionary subsystem.

To eliminate SSR's, objects and the SSR's that bind them are used to create object families (an object family is the set of objects inter-related by an SSR graph) and object combinations (an object combination is a subset of objects that may exist at one time as part of an object family instance, i.e. it defines a particular combination of SSR choices). The algorithm used to determine these is highly recursive and quite complex (maybe another day!). Each object family has one or more root objects (objects which never participate as subtypes) and one or more object combinations. The name of each object family can be derived from it's root objects and in many cases there will only be one. The name of each object combination can be derived from it's leaf objects (objects which never participate as supertypes).

In the new design, an object instance reference is actually an object family instance reference with an object reference for type qualification. While an object family instance held as a row within a repository table contains the cells for all potential attributes that an object family instance may have. It also has a current object combination indicator allowing us to determine which cells are currently accessible.

Migration now involves updating various cells in a row along with a simply change of object combination indicator. To determine if a particular object family instance reference is still valid one only has to check whether the current object combination indicated includes the object used as the qualifier within the reference. When an row is marked as deleted, depending on a setting associated with any reference holders (i.e. cells in other object instance family rows) we can now either: make reference holders invalid since they hold an invalid reference, automatically remove the reference from any holders allowing the holders to remain valid, or attempt to delete any reference holders to fix the situation.

Cast operations between different objects are easily performed by creating a new object reference containing the same object family instance reference but with a different object qualifier. Obviously, a cast is statically invalid between objects in different families, and dynamically invalid to an object not included in the current object combination indicated by the referenced object family instance. The scheme that I have adopted cleanly eliminates all SSRs without losing the benefits. The primary disadvantage is that each table row is now potentially much bigger than it needs to be. However, memory in my experience is not a resource in short supply.

Some graphic examples would have been nice here but I don't want to spend all day on this blog. Hopefully, I'm managed to convey the basic approach I'm using to eliminate subtype-supertype relationships (which are difficult to implement without recursive algorithms everywhere) from the OOA Tool II architecture by adding object families and combinations (which are easy to implement with simple lookups).

Week 11 of 2010

This week was very productive, focused mainly on the new Data Management domain at the heart of the new OOA Tool II architecture. I had some interesting thoughts on how OOA Tool has evolved and how it will evolve in the future:

• OOA Tool I (previously labelled OOA Tool 1.0) was incrementally prototyped. The approach allowed me to develop a rich OOA of OOA going beyond what was originally specified in OOA91 and OOA96, in step with a tool that supported it. However, over the last year I have spent a lot of time trying to make what should be simple changes work. Prototypes are just not scaleable or maintainable beyond a certain size and OOA Tool I has 145,000 lines of compact Java. However, without the knowledge gained from this prototype I would never have been able to develop a suitable architecture for the next version.
• OOA Tool II will comprise a completely generic configurable core (the Data Management domain) driven by XML dictionary/repository files for OOA of OOA, Work Product and GUI domain metadata/data. This version will include a working simulator and translator. It will allow me to develop one or more working software architectures (Shlaer-Mellor RD style).
• OOA Tool III (in the future) will use archetype templates to generate custom business objects and editor forms with a much reduced need for the previous generic configurable core. This version should be very fast (not that speed has been a problem so far) and may even include a web interface since we could generate active web content rather than traditional Java code. This version will fully demonstrate the power of Shlaer-Mellor OOA/RD, i.e. a code generation tool built using code generated from itself.
• OOA Tool IV? I'm not sure what major architectural improvement would drive this version. Perhaps a knowledge based query driven design assistent developed using the previous version could be used to automatically generate the necessary RD mappings to a generic software architecture. This is what I originally imagined OOA Tool would evolve into - an OOA/RD CASE tool with AI support for OOA/RD.

Now to progress on OOA Tool II:

• Now that I have dropped J2SE 5.0 support, I can take advantage of bundled features of Java SE 6. Loading OOA project files with Java SE 6's built-in XML parser is extremely easy. The first task was to load the OOA of OOA domain and create a suitable dictionary for the Data Management domain. Once created, this can be saved to a XML dictionary file for faster loading later. Last week, I mentioned that I would probably hand-code these dictionary files. However, it has become obvious that the process can be automated (with some hints not captured by OOA Tool at present) allowing future OOA of OOA changes to be deployed to OOA Tool II quickly. This involves mapping Objects, Attributes and Relationships to Node Types, Subnodes and Property Types (I will discuss this mapping in the future since it is a design problem for anyone generating code from Shlaer-Mellor OOA models).
• The second task now that the dictionary (i.e. metadata) is populated (or at least on the way to being populated) from the OOA of OOA domain, is to populate the repository from one or more project files using the OOA Interchange Format. Since I have already been using Java's built-in XML parser to do this task - no problem here. Except that the mapping from XML element/attributes to node/property types (in my dictionary) is a quirky one since the OOA Interchange Format was developed by hand. Future versions of the format will be generated automatically from the dictionary which is itself generated from the OOA of OOA.
• I have also extended the Data Management domain to include a Data Management Language that is a cross between an Action Language and an Archetype Language. This is going surprising quickly. I hope to develop the final Action/Archetype Languages as subsets of this language (maybe with a different syntax) making the simulator and translator relatively easy work. I will publish the syntax for this in the future.
• I haven't started working on the generic GUI yet since I first needed to be able to load metadata and data to browse. However, I will create a basic GUI domain especially for menu controls since OOA Tool I is mostly driven by menu controls and it would be nice for these to be configuration driven. The main components that I need are a tree browser and form editor. The tree browser is mostly generic in OOA Tool I. However, I want to incorporate more drag-and-drop functionality in OOA Tool II. Form editors in OOA Tool I were mostly hand-coded and this will be one of the major improvements of OOA Tool II since I will use a completely generic form editor. Obviously, I need sufficient information in my metadata to configure these. However, final validation of constraints is left to validation checking functions in business logic plug-ins (most OOA of OOA objects will have a hopefully small business logic plug-in). It should be possible to generate this logic automatically from an OOA model if all mathematically dependent attributes and relationships are coded in the OOA model (which they are not at present).

Obviously, the mythical build 014 for OOA Tool is on hold at present. However, coding of the OOA Tool II core is going fast enough that it will probably be the basis for the next build. I would estimate the work taking a few months. The difficult issues here will relate to moving across and testing all of the OOA of OOA business logic (constraints and derived calculations).

Week 8/9/10 of 2010

I have lots to say this week! I've been remiss over the last few weeks (i.e. no blogging) because I have had so many new ideas that I didn't want to sit down and try to explain them until the ideas had more fully matured. Some of these ideas have gone in unexpected directions and so now I have to sit down and do some explaining. Each of the different areas that I have been working on is numbered below.

(1) I evolved the Diagramming domain into a more complete Work Product domain with Diagram, Table and Template subsystems (not all done yet). The mission statement for the new domain covers work product information that needs to be persisted along with OOA of OOA information in model files. This is distinct from what is needed to display and change the work products in a GUI. This domain will be published since it forms part of the OOA Interchange Format.

(2) I also evolved the Naming domain to Name Management then to Text Management and finally to a Data Management domain with Dictionary, Repository, Change Control and Text subsystems. This domain will now be proprietary in nature but will have an open interface for user supplied language resources (e.g. French resources).

The Dictionary subsystem will allow notation and locale (language, country etc.) specific titles and icons to be loaded from Java resource files. The dictionary itself will now be loaded from an XML file. This XML file will initially be hand coded for OOA Tool but will later be generated from the OOA of OOA via an archetype template. Yep, I'm moving towards a self-generated OOA Tool.

The Repository subsystem with include built-in file system support that will allow all files to be loaded either from a local or networked hard-drive or over the internet. This will later be expanded to support database persistence and shared access.

The Change Control subsystem breaks down all changes into atomic reversible changes which are bundled into reversible transactions. Complete validation checks are then performed before transactions are committed. The repository will only save committed transactions. Users will be able to cancel a transaction or undo previous transactions (in order obviously). They will also be able to redo transactions etc. This will be a massive change from what is currently supported in OOA Tool.

The Text subsystem expands on the previous Name Management domain. I may separate this subsystem back out in future.

(3) I added concepts of Empty State, Deleted State, Creation Transition, Deleted Transition, Deleted Event and Communication Path to the State Model subsystem of the OOA of OOA. These changes were the result of a better understanding of what is needed by the Table subsystem of the new Work Product domain and what is needed to support Shlaer-Mellor and Executable UML notation.

It also reflects an important shift from user controlled memory management to automatic garbage collection, i.e. Deletion State is where user requests a deletion while Deleted State is a logical state after which deletion has occurred via garbage collection (a state which has meaning in the model but not at run-time). I personally don't believe it is possible for users to safely manage memory allocation and deallocation. For this reason, OOA10 reflects this assumption. You can still create C archetypes with memory allocation and deallocation logic but executing a delete object instance Action Language statement does not immediately delete an object instance. It only requests that a deletion should be carried out once all references to the object instance are cleared. However, users will not be able to search for a deleted object instance or navigate to such an instance but there may be any number of events carrying references to a deleted object instance (which we can't delete without unknown repercussions). There may also be actions being executed in parallel with access to deleted object instances (which would be impossible to delete).

(4) I have also been thinking about how to commercially manage reusable assets using public/private key encryption. Service domain suppliers will have their own public/private keys for their products (packaged as model files). They will securely distribute the public keys to licensed users. They will then encrypt the associated products (i.e. model files) using their matching private keys allowing the products to be openly distributed across the internet. Users download the products over the internet but have to manually enter any public keys into OOA Tool to access the products. OOA Tool will decrypt as required. This approach also allows users to verify the source of the products since public/private key encryption can be certicate based.

This approach can be extended to allow controlled user access (without having to securely distribute public keys) using an OOA Tool supplied public/private key combination. OOA Tool will include it's own public key hidden within itself while the private key will be made available to suppliers on request. Each imported domain will then be assigned an access level (from most restrictive to less restrictive):

• View - View domain as black box only allowing user to see control reception points, e.g. synchronous services, external events etc.
• Simulate - View domain as black box but allow OOA Tool simulator to access whole of domain for simulation purposes (users can't trace into domain making reverse engineering difficult).
• Translate - View domain as black box but allow OOA Tool simulator and translator to access whole of domain (obviously, output files from translator make reverse engineering easy).
• Edit - View domain as white box giving users full access.

The OOA Tool public/private key can be used as well as the the supplier's own public/private key to provide a flexible solution for commercial distribution of OOA assets.

(5) Finally, I have come to realize that the architecture of OOA Tool 1.0 is fatally flawed since it is not scalable or maintainable. The new Data Management domain will form the basis of future OOA Tool 2.0 development. The OOA Interchange Format will evolve in a controlled fashion so that users will be able to load old models. I may also continue with the old infrastructure for a few more builds. However, I will start to migrate stuff over as soon as the new stuff is stable. Since the Data Management domain will deliver a majority of OOA Tool 2.0's functionality, I have decided to make it a proprietary domain and not publish the domain's model. However, the OOA of OOA and Work Product domains will continue to be published and obviously the OOA Interchange Format will continue to be completely open. The new architecture should allow me to create changes in the OOA of OOA and rapidly roll them out. It should also allow me to make significant generic changes in the GUI without having to manually update large amounts of model specific GUI code.

That's all for now. Comments always welcome.

Week 7 of 2010

I continued working on the Recursive Design subsystem (follow this link if you want to understand what is discussed below!) and it's implementation within OOA Tool this week. I have moved bridge mappings, wormholes and control reception points into the Operation subsystem. This makes the RD subsystem much more managable and makes room for future growth, e.g. additional types of reusable assets will probably be added.

I replaced Domain IDs (Dn), Bridge IDs (Bn) and Layer IDs (Ln) with Asset IDs (An). This change means that bridge links within Domain Charts now use Asset IDs rather than Bridge IDs. However, I convert Bridge IDs when a model is loaded to maintain backwards compatibility. This also allowed me to change Data Type ID format from DTn to Dn.

Domain Activity is now Asset Activity allowing bridge and layer activities to be captured in a Project Matrix. I also renamed Activity to Task Activity to avoid confusion. However, to keep Project Matrix size down, columns are now only added when there is at least one activity in a column. Thus, they now look very empty to begin with. Note that, activities are project specific, i.e. are not loaded from imported assets.

Domain names are an interesting issue. They are currently unique within a project and are used as domain shape names within Domain Charts. However, imported domains load their names from an external project and thus may break uniqueness. This can be avoided by adding an External Name attribute to Imported Asset. However, you then have two names which may get out of step in a significant way. The alternative is to drop the uniqueness constraint and allow domain names to be duplicated. Since we now use domain role names within bridge mapping and domain observer code, this shouldn't be a big issue. I would need to change domain shape names to use Asset IDs. Domain Chart diagrams can also qualify duplicated domain names using Asset IDs to avoid confusion (if necessary). I'm 95% sure that I should choose the latter solution. Arguments to the contrary are welcome.

A similar issue concerns the multiplicity of bridge associations, i.e. whether each client/server domain pair can have at most one bridge. This made sense when all assets being bridged were local to a project. However, one may want to import two domains along with a existing bridge between the two domains, yet also add additional bridge mappings using an additional local bridge. This changes the multiplicity of R7 (in the RD OIM referenced above) to many. One could even imagine creating two local bridges within a single project to facilitate reuse by separating core bridge mappings from application specific bridge mappings. Making this change seems to be well justified so I will go ahead and change this next week.

I also downloaded the latest JDKs from Sun's website. I am now using Java SE 6 build 16 (JDK 1.6.0_16) for development and testing purposes. I also downloaded and tested Java SE 7 beta build 76. This beta still can't display HTML tables correctly so I can't recommend it yet. I also attempted to download J2SE 5.0 (JDK 1.5.0_22) but have gotten no response from Sun from a form request I submitted. It has reached end of life but I was still hoping to support it until Java SE 7 is finally released. Since it can't be downloaded anymore, I have decided to drop support for J2SE 5.0 from the next OOA Tool build. That means that I can now start using features that where added in Java SE 6 and remove numerous workarounds for J2SE 5.0 bugs (mostly GUI related). The next build will use the new class file format (50.0) so it definitely won't be runnable using J2SE 5.0. Speak now if anyone has issues with this decision, e.g. if you are running OOA Tool on a platform without Java SE 6 support.

Week 6 of 2010

I'm slowly getting back into things this week. However, I haven't been focusing on the priority one item which is to get Build 014 released. Instead, I have been trying to sort out a workable design for high-level reuse in the Recursive Design subsystem of the OOA of OOA. Enabling reuse of service domains and software architectures has always been a primary objective of OOA10 and OOA Tool. However, the current version of OOA Tool only supports reuse via copy and paste.

I've been working on a few ideas for reusing domains for some time. The main issue that has stopped me implementing an approach has been how to resolve parent links from a domain to the enclosing project, many of which are GUI related. I originally wanted users to be able to open multiple projects within OOA Tool where one project reuses one or more domains from another project. I envisaged a particular domain having only one actual instance at any one time. However, this leads to all sorts of linkage problems that could not be resolved without increasing the complexity of the current implementation considerably. Thus, I decided to abandon the goal of having only one instance of a domain at any one time. When a domain is imported into another project and separately loaded then it will be copied into that project but will not be persisted with the project. Users will also not be able to edit imported domains. They will be able to reload them. Automatic synchronisation may be possible in the future but will probably be processor intensive.

I also decided to broaden reuse across a number of high-level assets including domains, bridges and layers. I will probably add populations and transformations (simulations and translations) to this list in the future. This was done by defining a new Asset object which generalizes high-level reusable components. To reuse an asset within a project, users must first define an External Project allowing the location of the asset to be specified, then an external asset can be defined which imports the desired asset from the external project. The asset can then be loaded from the specified location, e.g. the associated information model can be loaded. Users can also load the whole of the external project into OOA Tool. In this situation, any external assets which are loaded (or reloaded) should be loaded from the current instance of the external project which may have been edited since it was loaded.

Bridges and layers introduce an additional resolution issue when they are reused. A simple mechanism for importing a bridge would be to ensure both the client and server domain were already imported from that external project. However, either or both domains may have already been imported from another external project. Alternatively, a newer or compatible version of one of the domains may already be defined in the project. The approach that I have decided to adopt for now is to require users to explicitly resolve client and server domains when they import a bridge from an external project. If the domains are incompatible with any bridge mappings then appropriate errors will be flagged separately. The same applies to layers, users will need to explicitly resolve all connected domains when a layer is imported.

This approach provides considerably flexibility but leads to an interesting problem. How do we identify client and server domains within bridge mappings, e.g. when we want to qualify a data type. I had envisaged using domainName@ prefixes as needed within bridge mapping code. However, we may want to resolve a client or server domain to a domain with a different name. The solution I have adopted is to define domain role names and use those within bridge mappings instead, i.e. users will now use client@ or server@ prefixes instead. This solution can also be adopted in domain observer code assuming each connection defines a unique role name.

OK, so I have a workable design for domain (and other asset) reuse within projects. I have started implementing the above design and another week or so should see this work finished. This work needed to be done sooner rather than later anyway. Comments always welcome but I prefer them in English!

Week 3/4/5 of 2010

I haven't been able to spend much time on this project over the last few weeks. I've done some modelling and fixed a few OOA Tool bugs but I haven't made substantial progress towards getting the next build released. However, some of the bugs might be quite annoying for anyone using the last ALPHA release so I have done another release which can be downloaded from the following link:

2010-02-08_OOATool014_ALPHA2 (8.33 MB)

Week 2 of 2010

My enthusiasm still seems to be on holiday at the moment. It better come back soon or I'm going to be pissed! I spent most of this week doing admin stuff and going over last year's progress in detail. I've renamed OOA09 to OOA10. I also moved my whole website onto a new server at www.easily.co.uk to take advantage of a free bandwidth offer. I reinstalled the forum from scratch using the latest version of the software. However, within 5 minutes of installing it I already had several bots trying to register user accounts. I've now disabled user registrations. I would love to get the forum going but anyone who wants to register will now have to email me first.

Below is a summary of last year's progress:

January

• Added Project Matrix support.
• Added Metamodel Population support.

February

• Released Build 013 of OOA Tool.
• Created an OOA model of fUML.
• Published technical note on Attribute Classification.
• Referential and polymorphic attributes now calculated automatically in model populations.
• Abstracted concept of operation and parameter out of OOA of OOA's process model subsystem.
• Adopted Nimbus (from Java SE 6) as default look and feel for OOA Tool.

March

• Added Object Instance Type support.
• Redesigned and reimplemented preferred types.
• Worked on Action Language syntax and semantics.
• Adopted policy of automatic creation of supertypes so that users do not need to relate or unrelate subtype-supertype relationships any more.
• Predefined type visibility within tree browser now depends on whether they are referenced within model.
• Added Event Reference Type support.
• Mathematically dependent referential attributes which link to mathematically dependent relationships added.
• Simplified default value design on attributes and data items.

April

• Major rewrite of attribute's derived fields which are pretty complex.
• Published technical note on Custom and Automatic Key Lettering.
• Added Enumerated Subtype Type and Enumerated State Type support.

May

• Added four new statements to Action Language which aren't defined in OAL: a declare statement, a switch statement, a migrate statement, and a delete event instance statement.
• Published technical note on Synchronous Communications between Domains.
• Published technical note on Asynchronous Communications between Domains.
• Prototyped wormholes and bridge mappings.
• Evolved concept of watchpoint mappings into domain observers.

June

• Published technical note on Semantic Shift.
• Explored use of declare statement in error handling.
• Considered introducing mathematically dependent objects but have held off for now.
• Identified need for operation owner and semantics of operation name ownership.

July

• Added data type unary/binary/dot operators as a new type of operation.
• Renamed Data Dictionary subsystem as Data Type subsystem.
• Identified clear role for process models as means of identifying concurrent data flows which is no longer a responsibility of the Action Language.

August

• Explored partial/complete ordering of objects and identified a potential use for aggregation and composition within Executable UML.
• Added Ordered Identifier support.
• First major refactoring of OOA Tool code - made ModelElement parents static, added deleted status, improved changed status and used Java generics to specific parent type.
• Allow users to control importing of objects and event destinations in various diagrams.

September

• Merged in some Java classes from a previous project: status bar component, console panel component, and improved event logging.
• Second major refactoring of OOA Tool code - added factory reference to all ModelElement classes using Java generics to control typing, and added view factory mechanism.
• Third major refactoring of OOA Tool code - encapsulated all Shlaer-Mellor and Executable UML terminology into OOA Dictionary component, added OOA Dictionary web page generator to that component.

October

• Added version control to OOA Dictionary and can generate OOA Dictionary web page within OOA Tool.
• Old icons recoloured and lots of new icons added.
• Now allow subtype-supertype relationships with only one subtype participant.
• Added automatic backup mechanism for model files.
• Modelled a new File System domain which will be bridged to the OOA of OOA (and implemented in OOA Tool) in the future.
• Added Layer concept to Recursive Design subsystem.
• Final attributes are now tagged with an "(F)" suffix in OIMs.
• Verb phrase concept is split into verb phrase before and verb phrase after to clarify meaning and allow generalization.
• Roles which appear within associated verb phrases are now highlighted on OIMs and class diagrams.
• Added concept of qualified verb phrase providing a more stable identification scheme for relationships compared with relationship IDs.

November

• Resigned Naming domain as Name Management domain, adding a Name Manager external entity to OOA of OOA encapsulating all the various types of name and qualified name used within the OOA of OOA.
• Created a detailed state model for name parsing.
• Added numeric word formatting support to Name Format and decided that numeric ID names within OOA of OOA should be padded by default.
• Developed data type to object bridging ideas from OOA of OOA to Name Management bridge - external types now allow a literal type to defined which allows external types to have default values, and external operators can be defined on external types which may override predefined operators such as equality.

December

• Added support for reference field navigation to OOA Tool GUI which was incorporated into all data type forms.
• Finished all outstanding tasks in the Data Type subsystem.

What it does not include is a working simulator or model compiler. The original goal for 2009 was to get OOA Tool out of BETA which requires working simulation and full translation. What went wrong?

The first thing that went wrong was that I didn't perform enough analysis of the Operation subsystem before I jumped into coding. This is an area that isn't well defined within Shlaer-Mellor or Executable UML, especially with regards to bridging operations. I was also determined to find a role for process models within the framework. The Operation subsystem also interacts with an Action Language subsystem that also isn't well defined with each tool vendor defining their own action language. The result was numerous iterations of ideas and a considerable amount of recoding.

The second thing that went wrong was a number of major refactorings of the OOA Tool code base. These required major effort to complete since the code base is already so large (almost 140K LOC excluding comments). They have improved the code considerably but each major refactoring left me pretty drained. Of course, if I was generating my model element classes from archetype templates and the OOA of OOA then these refactorings would have been much much easier.

The third thing that went wrong was that I made too many changes to the code base making me reluctant to release ongoing builds until a stable build was ready. The OOA Interchange Format and OOA of OOA then got out of step. This turned Build 014 (which has still not been released) into a year long effort rather than a 3 month effort. Not being able to release any stable builds has been a big downer and reduced external interest in the overall project.

So what now? I need to stabilize everything I'm doing. Get Build 014 released. Publish a number of partially complete technical notes that should have been published last year. Make my website notation switchable since I don't like mixing terminology within my web pages and I have no intention of giving up on Shlaer-Mellor notation even though most users are more interested into Executable UML notation. I then need to plan out future builds so that they are more timely. I would still like to get OOA Tool out of BETA this year but I would settle for a working simulator with a version controlled Action Language.

I'll end this week's report with a suggestion that you all have a browse of Kennedy Carter's new redesigned website. Hopefully, they will continue to expand the online content now that it has been refreshed.