I've spent the last 15 months working on a project that was particularly complex. One of the techniques I used in recent times to try to understand and manage that complexity was to use yFiles to draw dagrams that visually represent the system.
The first diagram I created was one that described the dependencies between modules in our system. We've been using Ivy to manage our module dependencies, so I wrote a script to parse the Ivy files and generate a graphml file that Ivy knows how to parse. Here's the result:
This diagram shows over 100 inter-dependant components that developers work with every day. The Ant build file controls every one of them in a single build. Its definitely complex. I won't go into why it should or shouldn't be this way right now. The important things to nore is that it is that way, and its extremely hard to understand the impacts of a change to any single component. Visualizing the dependencies just helps a bit.
The second diagram I created was one that I drew by hand. It shows all of the runtime components in the system and how they depend on each other whilst they are running. This diagram shows about 30 portlets, 60 EJBs, 15 JMS queues and a few 3rd party legacy systems.
This again was something that was very hard for most of the development team to visualise in their heads without some help. The nice thing about drawing it in yFiles is that it will lay out the diagram so that all of the components are clearly visible.
Update: I was contacted by someone from yWorks to ask if they could use these graphs as examples on their site. I've created some sanitised versions of the graphs for them. So you can download dependencies.graphml or architecture.graphml if you want to see more detail.
Comments
When I first saw your opening comment "I've been working" on a complex system, I thought "That's cute", then I scrolled down and say your diagrams... that is a hell of a system.
What does it do?
Posted by: Riyad Kalla | August 29, 2006 12:48 AM
I don't think I can be too specific unfortunately. I can tell you its a secure public facing web portal that integrates with a bunch of existing legacy systems at the back end. :)
Posted by: Marty Andrews | August 29, 2006 11:08 AM
I can't see any detail but I'm not sure that 1st diagram would help me. When everything is connected to everything else visualising that in a diagrams seems like it wouldn't be much benefit.
Posted by: Glen Stampoultzis | August 30, 2006 01:40 PM
It helps because some people believed that they weren't connected until I generated the diagram.
Posted by: Marty Andrews | August 30, 2006 04:51 PM
If the main use is to convince someone how complicated it was I'm sure that diagram would do it!
Posted by: Glen Stampoultzis | September 1, 2006 01:08 PM
Interesting to see how you addressed such a complex system. Does Ivy work well in such a complex environment (except your issue with subant you posted a whle ago)? Which version do you use? Do you have some feedback which could be useful for other users or to improve Ivy? And what about your script to generate the graph, can you share information about it?
It's a lot of questions, but I think your real world experience is very interesting and could benefit other Ivy users!
BTW, when you say "generate a graphml file that Ivy knows how to parse", don't you mean "that yFiles know how to parse"?
Posted by: Xavier Hanin | September 1, 2006 01:46 PM
Hi Xavier. Ivy has worked reasonably well. You'll see some postings from 'unclethehornet' (David Browne) in your forums, which was from one of my colleagues on the project. He's been providing some feedback and patches related to our work.
Posted by: Marty Andrews | September 1, 2006 05:51 PM
Very interesting and useful.
Posted by: Dmitry | March 29, 2007 09:57 PM