With the Decision Model and Notation (DMN) 1.0 approved by the OMG in December 2014, that news was quickly followed by an announcement from Signavio that its platform now “fully supports DMN”.
I’ve used the process editor extensively, so was excited (tragically) to see what the toolset had to offer.
My usual disclaimer….
Please note that this post expresses my own personal opinions; I’m not affiliated with the tool provider, nor have I been financially compensated in any way for evaluating the product or writing this review.
- My background in decision modelling
- How have I conducted the review?
- What did I need to complete the review?
- The decision modelling context
- Support for decision modelling
- Overall verdict
I’ve been a Business Analyst for 15 years (near enough) and an independent consultant in business analysis for 10. I came to decision modelling in 2010 and, over the last few years, I’ve been working with some large financial institutions, delivering decision modelling projects and embedding the capability to do decision modelling within those organisations. This has been using the notation and methodology of Sapiens’ The Decision Model (TDM), but the leap between that and DMN is not that big.
I could think of no better way to run the tool through its paces than to transplant my previous worked example straight into it. I rebuilt the process and attempted to recreate each of the decisions that I modelled as well. Rather than go step-by-step through that, I’ve collated my observations into a set of lists with screenshots to illustrate.
Just the latest release of the trial version of the editor, available from the Signavio site. So that was already a bonus – one tool; you just have to switch on the setting that allows DMN to be available.
**Update 20th Feb 2015** Since this post was first published, Signavio has been very keen to point out that there are additional features available in the non-trial version that address some of the points that I reference below. So I want to emphasise the use of the trial version of the software – I have no access to a more fully-fledged version: I can only evaluate what I’ve got, so I can’t vouch for the efficacy of these additional features.
I’m not going to dwell too much on the process side of things, because their process editor is so well established that it needs no real introduction. I’m a big fan of it and consider it so good that I don’t know why people even give IBM BlueWorks the time of day, given some of the basic things that are missing (no blackbox pools? Really?)
The process from the worked example, recreated in the Process Editor, is as follows:
The process is good to go and our decision modelling tasks are already there, denoted as BPMN business rule tasks. Next step is to see how the tool works and to do that, I was looking for a few key things:
- Usability – the saying “a User Interface is like a joke: if you have to explain it, it’s not that good” rings truer with each passing app-driven day. But I have a very personal interest in other aspects of usability – I’ve had wrist problems for a few years now, so I (literally) physically detest too much mouse usage, so I really want to see how Decision Tables can be built in comparison to using tools like Excel.
- DMN spec coverage – it claims full support; there are three levels of conformance within the DMN spec, but the press release doesn’t go as far as to state what ‘full support’ actually entails.
- Decision modelling methodology – as the OMG Spec is a standard and as such cannot have any methodology associated with it, I was very keen to understand what, if any, methodology was going to be put into the tool. I’ve mentioned in other posts at great length how easy it is to do decision modelling badly, which is what makes the structural and integrity principles of TDM so beneficial.
- Integration – one of the things that is likely to set a tool apart is its integration with the other components that organisations are already using, particularly speaking as a Business Analyst.
As with the review I did of the BiZZDesign ‘The Decision Modeler’ tool, I’m going to provide a notional score (my gut feel; nothing too scientific) and then hopefully explain how I got there, but the whole thing is going to be based on how well it supports that worked example, because the purpose of that piece in the first place was to give a fairly well-rounded view of the specification.
Verdict: diagrams and tables simple to create and feels at home with the suite, but there are a few tweaks that need to be made to some of the basics.
On the whole, it’s simple to navigate although there’s a lot of double-clicking in table cells to change things like the Hit Policy, or create an expression header. Tabbing through the table takes my hand away from the mouse, which is good, although I would like to be able to enter some of the information by hitting the space bar to activate the cell, or something similar, rather than tabbing into the cell and then having to click to get into it.
Creating diagrams is easy to do, particularly if you are used to the Signavio interface anyway and [SPOILER ALERT] you can only connect those elements that are allowed by the spec, which makes the graphical bit easier to learn on the job. The two diagrams below show the two decisions from the process:
The Decision Tables themselves are also straightforward to create, but it would be good if the Decision Table automatically took the name of the decision in the process. An example of one of the tables (check the Decision modelling methodology section for why the conclusions for rows 3 and 4 are blank) is as follows:
It has to be said that there are some odd quirks, though:
- No text wrap in the Decision Table – this may sound minor, but it’s odd that you have to pull the column out in order to see the whole of an expression name. You don’t see it when you hover over and there doesn’t seem to be a way to wrap it in the cell. In addition, there’s no scrollbar in the editor window, which is a bit annoying. You can see what I mean in the example above.
- You can’t test ranges of numbers – so if you want to test that a value is between two numbers, I couldn’t seem to find a way to do it, other than to put two columns in with the same Input Expression and have one as greater than and the other as less than.
- Boolean: True, False, or.. div xmins? – if you’ve got a Boolean Input Expression, when you select the operand of the cell, in addition to the true/false, you also get a rather strange value pop up in the field by default:
- The disappearing zero – this one was really weird. For some reason, enter a zero as a value in a cell and when you click out, it’s gone. Click in, it’s there, click out, it’s gone:
Minor bugs really, other than the number range part, but suggests more testing could have been done, because it gives it a sloppy feel in parts.
DMN spec coverage
Verdict: covers a good range of the Decision Table types and also supports the use of Literal Expressions. However, doesn’t provide any of the underlying attributes that you might hope for.
In summary, here are my findings:
- Full coverage of the Hit Policies – Hit Policy is selected through a double-click of the top-left corner of the Decision Table, presenting you with the available Hit Policies and indicating whether the policy creates a Single-Hit or Multi-Hit table:
- Decision Tables in rules-as-rows format – rules-as-rows is likely to be the one that finds most frequent use in the modelling community. Not sure ‘full support’ can be claimed on the basis of this, though – there are other formats: rows-as-columns; cross-tab, but in all honesty, I expect those to be ‘nice-to-haves’ in most tools.
- Option to put logic into a Business Knowledge Model or a decision – both elements can be added to the diagram and the editor allows the logic to be put in either.
- Modelling in both Decision Tables and Literal Expressions supported – so you can add a Decision Table, or you can express yourself via the medium of natural language. There is no enforcement of the FEEL language for these expressions, but I don’t really see that as a problem – in the conceptual and logical stages of modelling, the fact that you can pick from a Decision Table or a Literal Expression is good enough:
- No merging of cells – I don’t see this as a downside, particularly, but that’s because TDM doesn’t support merging cells where the Input Expression is the same in multiple rows. Merging cells immediately creates maintenance problems in the same way it does in Excel. That said, it gets done in Excel all the time, so some people might think this is a minus point.
- Multi-Hit aggregator supported but not visually – I think this adds great value to the readability of the Decision Table. You’ll see from the Credit Score Table example that I did in Excel that there is an aggregator of ‘Sum’ (rather than the default ‘Collect’). Showing this visually will have definite benefits so it’s a shame it’s missing from this version.
- No overall DRD view – so what I mean by that is that there is no graphical view of the whole decision domain. Each diagram is self-contained. My worked example only has the Input Data ‘Application’ tying the two together, but it’s still a useful view. From experience, there are stakeholders that will actually eschew process in order just to understand on the decisions being made. So without a holistic DRD view, a process context will always be required.
Verdict: has basic validation in terms of the allowable connections between different element types, but nothing in the way of logic validation. Whilst the diagram recognises connections between elements, there is nothing to link the Output Expression of one Decision Table to the Input Expression of another.
Well, where to start? In summary, there is no methodology built into the tool. It may be unfair of me to judge the product using methodology as a criteria; there are many marketing strategies that could justify the release of something in order to be one of the first to market and I suspect a big part is to make the right noises with their existing customer base.
In my own opinion though, you don’t rush a car to market so quick that you leave the engine out.
My observations are as follows:
- Diagram only allows those elements to which you may connect – not technically methodology, because the connection rules are part of the spec, but there’s less rosy news further on, so I wanted to start with a win. It ensures you can’t connect a piece of Input Data to another piece of Input Data. Essentially the same type of shape connectivity validation built into the process modeller.
- Validation of data types – when you specify the data type of your expression, there is validation to ensure you’re putting something allowable in. For enumerations, you can only select from values that have been specified. For numbers, you can only enter valid numbers:
- No prevention of rule conflicts – as an example:
So the problem here is that rows 1 and 4 overlap. Columns 2 and 3 can be any value in row 1, which means that, for a value of greater than or equal to 100000, you’d hit both rows, which would lead to two different conclusions. Which, one would think, cannot be right. However, you can save and move on and no-one’s the wiser.
- Multiple selections allowed in the output expressions – this could be deliberate; DMN does support the idea of what it calls a ‘compound output’, that is, more than one output for a single rule. I don’t like it, personally, but it’s there. However, the decision table structure shown in the spec would therefore require two output columns, comme ca:
Not so in this tool, so it can make for confusing reading when you’re suitability decision could result in both a ‘Suitable’ and ‘Unsuitable’ for the same rule:
- No validation across logic within a single DRD – each piece of logic associated with an element is entirely encapsulated, but thus entirely isolated. So there is no check to see whether one decision makes sense in the context of another.
- No link between decisions – when an Input Expression is created, you get the option to select from any of the Input Data elements that are in the model. However, it doesn’t allow you to pick any of the other decisions as inputs. So whilst the diagram will show one decision requiring another, there is nothing in the underlying logic to actually support it. What I would expect is to see a list of decisions in addition to the raw Input Data. What this means in practice is that you’ll have to rekey the expression that is the output of the other decision and, without validation, or link to the dictionary (see Integration), then this could become a maintenance headache quite quickly.
- Unable to use the output expression of one decision as the operand of another decision – I’ve saved my favourite bugbear to last. This is the same thing I harped on about in the BiZZDesign review. I asked Signavio whether this was supported and got the response that “the graphical notation is not meant to be used in the way … described, i.e. invoking a business knowledge model in an output entry / output cell of a decision table”. I bite my thumb at you, sir. Why do I care about this so much? I’m delighted you asked:
- It’s how often it’s used in practical scenarios – I put an explanation in the BiZZDesign post using an example but I’ll be more generic here: let’s say I’ve got an outcome that can be reached using different combinations of the same conditions in 80% of cases; this would form the basis of my decision table. But supposing the remaining 20% use an extra 3 conditions that the rest don’t? Rather than make my Decision Table unwieldy, I pull that logic out into its own table and invoke it instead.
- I put it down to a lack of practical experience – harsh, but fair. It is in doing the do that you really start to understand these things. Like I said in the BiZZDesign review, I think missing this kind of thing is fairly fundamental, but I suspect the response comes from an academic interpreting the spec, not a practitioner that will actually need to be able to do the work. But what do I know?
What did this all mean in practice? I couldn’t complete many of the tables in my worked example. There are other configurations that I could have defined that would probably have produced the same results, but they wouldn’t have been as efficient. As I said above, I could be judging the tool unfairly if methodology was never part of its scope, but it’s mugs like me that have to use these things 🙂
Verdict: from an holistic analysis viewpoint, it fits really well with the rest of the tool from a diagrammatic perspective. Where it needs to improve is its integration with data, specifically the dictionary.
- Process can be linked to decision and vice-versa – so I can create a standalone DRD, or create one by going through a business rule task in the process. In the case of the latter, I can also link to one that has already been created.
- No link to the dictionary/data modelling – where a single tool will really come into its own is when it links together the process, decision, and data. BiZZDesign did this really well, in that a glossary entry used in a decision could be linked to a data model. Signavio has a dictionary but, unlike the process editor, there is no prompt when you start entering Input Data to any corresponding dictionary entries. The expressions need to be treated as a key asset in the same way as the decision itself; understanding those expressions is a huge part of understanding the rules. I imagine this is something that won’t be long coming, though.
Overall, I was disappointed; I stopped trying to build the model from my worked example when I realised how many things couldn’t be supported in the tool, mainly caused by a lack of methodology. As far as I can see, Signavio have put this tool to market to support the notation, which is fine, but not particularly practical. I can build a model that I know will hang together based on my experience, but that doesn’t help anyone tackling decision modelling for the first time.
So, as I’ve said so many times before, it will be easy to do decision modelling badly and unfortunately, this initial offering won’t change that. However, if you want to consolidate any decision modelling you’ve done in Excel and Visio and link them into your processes, then this will do the job. So an attractive proposition for existing clients, but not a game-changer. Yet.
What’s right with it?
It’s well integrated with the other elements of the package, as you would hope – you can create a BPMN diagram and link a decision to it, or you can create a standalone decision diagram. It provides support for the key components of the notation.
What’s wrong with it?
It clearly lacks any real underlying experience of practical decision modelling. Every post I ever do about DMN points to how easy it will be to create unwieldy, unmaintainable models because best practice principals are required. Not only does this not support certain key constructs (speaking from years of experience of building decision models), but its lack of validation means that you can happily create models that are entirely wrong. Whilst the diagram is comprehensive, the diagram does not have the same value as in BPMN – the logic components have to be right.
Stakeholders will express decisions in logic first, rather than diagrams, because that’s how they’ve specified rules all this time. So whilst the diagram will pull it all together, no-one’s going to sign ANYTHING off until they know it works, whether that’s on paper or in the target application.
Verdict: good support for the types of Decision Tables and only allows the right elements to be connected to each other, but, with with no other methodology or validation, it’s far too easy to create models that are fundamentally wrong.