Cometh the hour, cometh [Signavio]

With the Decision Model and Notation (DMN) 1.0 approved by the OMG in December 2014, that news was quickly followed by an announcement from Signavio that its platform now “fully supports DMN”.

As Alan Partridge once said when interviewing France’s second-best racing driver: “I’ll be the judge of that” 🙂

I’ve used the process editor extensively, so was excited (tragically) to see what the toolset had to offer.

My usual disclaimer….

Please note that this post expresses my own personal opinions; I’m not affiliated with the tool provider, nor have I been financially compensated in any way for evaluating the product or writing this review.

Contents


My background in decision modelling

I’ve been a Business Analyst for 15 years (near enough) and an independent consultant in business analysis for 10. I came to decision modelling in 2010 and, over the last few years, I’ve been working with some large financial institutions, delivering decision modelling projects and embedding the capability to do decision modelling within those organisations. This has been using the notation and methodology of Sapiens’ The Decision Model (TDM), but the leap between that and DMN is not that big.

Having learnt DMN from the specification, I went about doing a worked example that was published back in January 2014, followed by my view on how you interchange between TDM and DMN.

^top


How have I conducted the review?

I could think of no better way to run the tool through its paces than to transplant my previous worked example straight into it. I rebuilt the process and attempted to recreate each of the decisions that I modelled as well. Rather than go step-by-step through that, I’ve collated my observations into a set of lists with screenshots to illustrate.

^top


What did I need to complete the review?

Just the latest release of the trial version of the editor, available from the Signavio site. So that was already a bonus – one tool; you just have to switch on the setting that allows DMN to be available.

**Update 20th Feb 2015** Since this post was first published, Signavio has been very keen to point out that there are additional features available in the non-trial version that address some of the points that I reference below. So I want to emphasise the use of the trial version of the software – I have no access to a more fully-fledged version: I can only evaluate what I’ve got, so I can’t vouch for the efficacy of these additional features.

^top


The decision modelling context

I’m not going to dwell too much on the process side of things, because their process editor is so well established that it needs no real introduction. I’m a big fan of it and consider it so good that I don’t know why people even give IBM BlueWorks the time of day, given some of the basic things that are missing (no blackbox pools? Really?)

The process from the worked example, recreated in the Process Editor, is as follows:

BPMN diagram using Signavio's Process Editor

BPMN diagram using Signavio’s Process Editor

^top


Support for decision modelling

The process is good to go and our decision modelling tasks are already there, denoted as BPMN business rule tasks. Next step is to see how the tool works and to do that, I was looking for a few key things:

  • Usability – the saying “a User Interface is like a joke: if you have to explain it, it’s not that good” rings truer with each passing app-driven day. But I have a very personal interest in other aspects of usability – I’ve had wrist problems for a few years now, so I (literally) physically detest too much mouse usage, so I really want to see how Decision Tables can be built in comparison to using tools like Excel.
  • DMN spec coverage – it claims full support; there are three levels of conformance within the DMN spec, but the press release doesn’t go as far as to state what ‘full support’ actually entails.
  • Decision modelling methodology – as the OMG Spec is a standard and as such cannot have any methodology associated with it, I was very keen to understand what, if any, methodology was going to be put into the tool. I’ve mentioned in other posts at great length how easy it is to do decision modelling badly, which is what makes the structural and integrity principles of TDM so beneficial.
  • Integration – one of the things that is likely to set a tool apart is its integration with the other components that organisations are already using, particularly speaking as a Business Analyst.

As with the review I did of the BiZZDesign ‘The Decision Modeler’ tool, I’m going to provide a notional score (my gut feel; nothing too scientific) and then hopefully explain how I got there, but the whole thing is going to be based on how well it supports that worked example, because the purpose of that piece in the first place was to give a fairly well-rounded view of the specification.


Usability

Score: 6/10

Verdict: diagrams and tables simple to create and feels at home with the suite, but there are a few tweaks that need to be made to some of the basics.

Review

On the whole, it’s simple to navigate although there’s a lot of double-clicking in table cells to change things like the Hit Policy, or create an expression header. Tabbing through the table takes my hand away from the mouse, which is good, although I would like to be able to enter some of the information by hitting the space bar to activate the cell, or something similar, rather than tabbing into the cell and then having to click to get into it.

Creating diagrams is easy to do, particularly if you are used to the Signavio interface anyway and [SPOILER ALERT] you can only connect those elements that are allowed by the spec, which makes the graphical bit easier to learn on the job. The two diagrams below show the two decisions from the process:

Applicant Demographic Suitability DRD from the worked example, recreated in the DMN Editor

Applicant Demographic Suitability DRD from the worked example, recreated in the DMN Editor

Second of the two decisions from the process

Second of the two decisions from the process

The Decision Tables themselves are also straightforward to create, but it would be good if the Decision Table automatically took the name of the decision in the process. An example of one of the tables (check the Decision modelling methodology section for why the conclusions for rows 3 and 4 are blank) is as follows:

Example Single-Hit DMN Decision Table created in Signavio

Example Single-Hit DMN Decision Table created in Signavio

It has to be said that there are some odd quirks, though:

  • No text wrap in the Decision Table – this may sound minor, but it’s odd that you have to pull the column out in order to see the whole of an expression name. You don’t see it when you hover over and there doesn’t seem to be a way to wrap it in the cell. In addition, there’s no scrollbar in the editor window, which is a bit annoying. You can see what I mean in the example above.
  • You can’t test ranges of numbers – so if you want to test that a value is between two numbers, I couldn’t seem to find a way to do it, other than to put two columns in with the same Input Expression and have one as greater than and the other as less than.
  • Boolean: True, False, or.. div xmins? – if you’ve got a Boolean Input Expression, when you select the operand of the cell, in addition to the true/false, you also get a rather strange value pop up in the field by default:
Slightly strange behaviour from the Boolean Input Expression

Slightly strange behaviour from the Boolean Input Expression

  • The disappearing zero – this one was really weird. For some reason, enter a zero as a value in a cell and when you click out, it’s gone. Click in, it’s there, click out, it’s gone:

A zero is entered as a value I want to test

A zero is entered as a value I want to test

And the zero is gone!

And the zero is gone!

Minor bugs really, other than the number range part, but suggests more testing could have been done, because it gives it a sloppy feel in parts.

^top


DMN spec coverage

Score: 6/10

Verdict: covers a good range of the Decision Table types and also supports the use of Literal Expressions. However, doesn’t provide any of the underlying attributes that you might hope for.

Review

In summary, here are my findings:

  • Full coverage of the Hit Policies – Hit Policy is selected through a double-click of the top-left corner of the Decision Table, presenting you with the available Hit Policies and indicating whether the policy creates a Single-Hit or Multi-Hit table:
Options for Hit Policy in the Decision Table editor

Options for Hit Policy in the Decision Table editor

  • Decision Tables in rules-as-rows format – rules-as-rows is likely to be the one that finds most frequent use in the modelling community. Not sure ‘full support’ can be claimed on the basis of this, though – there are other formats: rows-as-columns; cross-tab, but in all honesty, I expect those to be ‘nice-to-haves’ in most tools.
  • Option to put logic into a Business Knowledge Model or a decision – both elements can be added to the diagram and the editor allows the logic to be put in either.
  • Modelling in both Decision Tables and Literal Expressions supported – so you can add a Decision Table, or you can express yourself via the medium of natural language. There is no enforcement of the FEEL language for these expressions, but I don’t really see that as a problem – in the conceptual and logical stages of modelling, the fact that you can pick from a Decision Table or a Literal Expression is good enough:
Example of how the Literal Expression as an alternative to a Decision Table

Example of how the Literal Expression as an alternative to a Decision Table

  • No merging of cells – I don’t see this as a downside, particularly, but that’s because TDM doesn’t support merging cells where the Input Expression is the same in multiple rows. Merging cells immediately creates maintenance problems in the same way it does in Excel. That said, it gets done in Excel all the time, so some people might think this is a minus point.
  • Multi-Hit aggregator supported but not visually – I think this adds great value to the readability of the Decision Table. You’ll see from the Credit Score Table example that I did in Excel that there is an aggregator of ‘Sum’ (rather than the default ‘Collect’). Showing this visually will have definite benefits so it’s a shame it’s missing from this version.
  • No overall DRD view – so what I mean by that is that there is no graphical view of the whole decision domain. Each diagram is self-contained. My worked example only has the Input Data ‘Application’ tying the two together, but it’s still a useful view. From experience, there are stakeholders that will actually eschew process in order just to understand on the decisions being made. So without a holistic DRD view, a process context will always be required.

^top


Decision modelling methodology

Score: 2/10

Verdict: has basic validation in terms of the allowable connections between different element types, but nothing in the way of logic validation. Whilst the diagram recognises connections between elements, there is nothing to link the Output Expression of one Decision Table to the Input Expression of another.

Review

Well, where to start? In summary, there is no methodology built into the tool. It may be unfair of me to judge the product using methodology as a criteria; there are many marketing strategies that could justify the release of something in order to be one of the first to market and I suspect a big part is to make the right noises with their existing customer base.

In my own opinion though, you don’t rush a car to market so quick that you leave the engine out.

My observations are as follows:

  • Diagram only allows those elements to which you may connect – not technically methodology, because the connection rules are part of the spec, but there’s less rosy news further on, so I wanted to start with a win. It ensures you can’t connect a piece of Input Data to another piece of Input Data. Essentially the same type of shape connectivity validation built into the process modeller.
  • Validation of data types – when you specify the data type of your expression, there is validation to ensure you’re putting something allowable in. For enumerations, you can only select from values that have been specified. For numbers, you can only enter valid numbers:
Warning that the user is entering an invalid number into the operand

Warning that the user is entering an invalid number into the operand

  • No prevention of rule conflicts – as an example:
Single-hit Decision Table with rule conflict in rows 1 and 4

Single-hit Decision Table with rule conflict in rows 1 and 4

So the problem here is that rows 1 and 4 overlap. Columns 2 and 3 can be any value in row 1, which means that, for a value of greater than or equal to 100000, you’d hit both rows, which would lead to two different conclusions. Which, one would think, cannot be right. However, you can save and move on and no-one’s the wiser.

  • Multiple selections allowed in the output expressions – this could be deliberate; DMN does support the idea of what it calls a ‘compound output’, that is, more than one output for a single rule. I don’t like it, personally, but it’s there. However, the decision table structure shown in the spec would therefore require two output columns, comme ca:
Example structure of Compound Output in a Single-Hit Decision Table

Example structure of Compound Output in a Single-Hit Decision Table

Not so in this tool, so it can make for confusing reading when you’re suitability decision could result in both a ‘Suitable’ and ‘Unsuitable’ for the same rule:

Single-hit table showing two conclusions in the same Output Expression cell

Single-hit table showing two conclusions in the same Output Expression cell

  • No validation across logic within a single DRD – each piece of logic associated with an element is entirely encapsulated, but thus entirely isolated. So there is no check to see whether one decision makes sense in the context of another.
  • No link between decisions – when an Input Expression is created, you get the option to select from any of the Input Data elements that are in the model. However, it doesn’t allow you to pick any of the other decisions as inputs. So whilst the diagram will show one decision requiring another, there is nothing in the underlying logic to actually support it. What I would expect is to see a list of decisions in addition to the raw Input Data. What this means in practice is that you’ll have to rekey the expression that is the output of the other decision and, without validation, or link to the dictionary (see Integration), then this could become a maintenance headache quite quickly.
  • Unable to use the output expression of one decision as the operand of another decision – I’ve saved my favourite bugbear to last. This is the same thing I harped on about in the BiZZDesign review. I asked Signavio whether this was supported and got the response that “the graphical notation is not meant to be used in the way … described, i.e. invoking a business knowledge model in an output entry / output cell of a decision table”. I bite my thumb at you, sir. Why do I care about this so much? I’m delighted you asked:
    • It’s how often it’s used in practical scenarios – I put an explanation in the BiZZDesign post using an example but I’ll be more generic here: let’s say I’ve got an outcome that can be reached using different combinations of the same conditions in 80% of cases; this would form the basis of my decision table. But supposing the remaining 20% use an extra 3 conditions that the rest don’t? Rather than make my Decision Table unwieldy, I pull that logic out into its own table and invoke it instead.
    • I put it down to a lack of practical experience – harsh, but fair. It is in doing the do that you really start to understand these things. Like I said in the BiZZDesign review, I think missing this kind of thing is fairly fundamental, but I suspect the response comes from an academic interpreting the spec, not a practitioner that will actually need to be able to do the work. But what do I know?

What did this all mean in practice? I couldn’t complete many of the tables in my worked example. There are other configurations that I could have defined that would probably have produced the same results, but they wouldn’t have been as efficient. As I said above, I could be judging the tool unfairly if methodology was never part of its scope, but it’s mugs like me that have to use these things 🙂

^top


Integration

Score: 6/10

Verdict: from an holistic analysis viewpoint, it fits really well with the rest of the tool from a diagrammatic perspective. Where it needs to improve is its integration with data, specifically the dictionary.

Review

  • Process can be linked to decision and vice-versa – so I can create a standalone DRD, or create one by going through a business rule task in the process. In the case of the latter, I can also link to one that has already been created.
How the process is linked to the decision in the editor

How the process is linked to the decision in the editor

  • No link to the dictionary/data modelling – where a single tool will really come into its own is when it links together the process, decision, and data. BiZZDesign did this really well, in that a glossary entry used in a decision could be linked to a data model. Signavio has a dictionary but, unlike the process editor, there is no prompt when you start entering Input Data to any corresponding dictionary entries. The expressions need to be treated as a key asset in the same way as the decision itself; understanding those expressions is a huge part of understanding the rules. I imagine this is something that won’t be long coming, though.

^top


Overall verdict

Overall, I was disappointed; I stopped trying to build the model from my worked example when I realised how many things couldn’t be supported in the tool, mainly caused by a lack of methodology. As far as I can see, Signavio have put this tool to market to support the notation, which is fine, but not particularly practical. I can build a model that I know will hang together based on my experience, but that doesn’t help anyone tackling decision modelling for the first time.

So, as I’ve said so many times before, it will be easy to do decision modelling badly and unfortunately, this initial offering won’t change that. However, if you want to consolidate any decision modelling you’ve done in Excel and Visio and link them into your processes, then this will do the job. So an attractive proposition for existing clients, but not a game-changer. Yet.

What’s right with it?

It’s well integrated with the other elements of the package, as you would hope – you can create a BPMN diagram and link a decision to it, or you can create a standalone decision diagram. It provides support for the key components of the notation.

What’s wrong with it?

It clearly lacks any real underlying experience of practical decision modelling. Every post I ever do about DMN points to how easy it will be to create unwieldy, unmaintainable models because best practice principals are required. Not only does this not support certain key constructs (speaking from years of experience of building decision models), but its lack of validation means that you can happily create models that are entirely wrong. Whilst the diagram is comprehensive, the diagram does not have the same value as in BPMN – the logic components have to be right.

Stakeholders will express decisions in logic first, rather than diagrams, because that’s how they’ve specified rules all this time. So whilst the diagram will pull it all together, no-one’s going to sign ANYTHING off until they know it works, whether that’s on paper or in the target application.

Score: 3/10

Verdict: good support for the types of Decision Tables and only allows the right elements to be connected to each other, but, with with no other methodology or validation, it’s far too easy to create models that are fundamentally wrong.

^top

Advertisements

7 thoughts on “Cometh the hour, cometh [Signavio]

  1. Nick, thanks for the detailed review on our DMN capabilities. Of course, I would have hoped for a better overall score 😉 But I am pretty sure that future assessments will look a lot better: We are working on the tool with full speed and have already addressed quite a number of points you raised.

    For instance, there is already logic verification for enums, numbers and boolean built in. It was simply not activated in the version you tested. The verification includes completeness checks and checks for absence of overlapping rules.

    Also, the integration regarding reusable data definitions (we leverage the Signavio dictionary for that) has been added with last weekends deployment (again, we simply need to activate that for your tenant).

    Looking forward to continuing the dialog.

    Best,
    Gero

  2. Nick – very informative and well written review, thank you for taking the effort to compile it!

    I assume from your review that the tool does not support simulation (execution) or model governance either.

    Your comments about tools that fail to support semantic integrity and best practices – tools which feel like they have been written from an academic analysis of the spec, rather than by experienced decision modelers – is absolutely spot on. I completely concur with that observation. Similarly your remark about lack of good dictionary support/integration. Fact model integration should be provided too, in my view.

    One question that occurred to me while reading your review is your use of the term ‘methodology’. From your illustrations, it seems you were actually referring to semantic and structural integrity/consistency of the models that Signavio produces and the extent to which Signavio checks this in the models it creates. This is rather different to ‘methodology’ as I understand it. This is a small point though.

    Thanks again for an interesting read.

    Best Regards,
    Jan.

    • Jan,

      let me reply to your comment. I think there is a slight misunderstanding about what the tool can do and what it cannot do.

      The product evaluated by Nick was the Signavio Process Editor, which is a Business Process Analysis tool that also includes, among many other things, support for DMN. As Nick described, it allows to create Decision Requirements Diagrams as well as decision tables. What it does not do is decision model verification, simulation, generation of executable rules, lifecycle management for models, reusable input data definitions and so on. Therefore, I agree, it is quite limited for decision modeling.

      There is a second product, based on the same technical platform as the Signavio Process Editor, which is specifically oriented at decision modeling. It goes well beyond simple diagram and decision table creation. It is a new product. Already today, it supports things like decision table verification, reusable input data and decision model governance. Over the coming weeks, you will see a lot of new functionality, such as generation of executable rules, for instance.

      Unfortunately, this additional functionality, which can only be found in the decision modeling product, was not part of the evaluation in this blog post.

      If you have already signed up for a trial of the Signavio Process Editor, we can simply switch the additional functionality on. In the future, any trial sign-up at Signavio will give you access to both products at the same time. So I hope that this will avoid misunderstandings about limitations of the offering.

      Best,
      Gero

    • Hi Jan,

      Thanks for your reply; Gero’s obviously given you a more detailed reply about the capabilities of the other tool that is available but, as I mentioned in my reply to the post on LinkedIn, I could only test what I had access to and Signavio didn’t point out, or make available, any other version whilst I was working on this post.

      In terms of the simulation activity, as Gero points out, there isn’t the capability in this tool. That said, I wouldn’t have been able to tell you, as I ran out of time with the trial version. I knew it would be a tall order in 30 days, with part of that being over Christmas, so I focussed purely on the modelling capability.

      As for your methodology point, then that’s a fair challenge – I was thinking in terms of normalisation as part of ‘methodology’ and it’s normalisation that leads me to structuring decisions in their own decision tables in the operand of a conclusion. I was also thinking in terms of the TDM structural and integrity principles, hence the observation on rule conflicts.

      Thanks

      Nick

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s