This thread follows on from the 'Artifact metadata and dependency management' thread - which was more about how to represent dependencies between resources in different repositories.

This thread will focus more on what additional information would be useful in this schema, related to the project lifecycle and methodology. This also ties into the documentation generation task, as the information can be used in any generated documentation.

We may need different levels of information, at the (1) project, (2) phase, (3) resource, (4) dependency, and (5) uri levels.

We could define some standard information in the 'tia' schema, but allow extensions that could be used in customised documentation generation templates.

(1) Project level, we need basic info, like project name and a description. Possibly something related to the stakeholders, and the project managers.

(2) Phase level, not sure whether this file should hold project management related information - but if so, then it might define timescales for delivery of a phase? It should atleast have a description, that can be used in the generated documentation for the phase.

(3) Resource level, will obviously want a description - so (for example) for a scenario, it should explain what the scenario represents. There should be information about the authors and contributors (people interviewed to obtain information used in the resource definition). If project management info is being recorded, then timelines could be defined - including review points and possibly signoff - with the details of the authorising party. Change history would also be required - this information could be used as part of a change management system later on, to track project/service revisions and how they impact (or are impacted by) other internal/external components.

(4) Dependency level, should have a description field. May want to define the action that should be taken if a dependency issue is detected(?). We may want some conditionality on the dependency(?)

(5) URI level - probably not much additional information is required, although the standard description should be allowed, possibly to explain why a particular location for the resource has been included.

A general issue will be how the information recorded in the tia is affected over revisions of the project and its artifacts. If review/signoff is recorded, then this needs to be tied to specific iterations. What information is explicitly recorded, and what information is subject to external version management.

Due to the varied locations in which this file may need to be used, it may be best to forget version management, other than as a way to track how the file contents changed over time. Therefore we need to consider how best to cater for project iterations and the impact it has on the information we record.

What should happen if there are two or more iterations occuring concurrently - i.e. a development branch, and a productisation branch. Should the same file be used to represent both, and the version information help to isolate which parts are specific to each version?

We need to consider some usecases, where issues may arise - for example, the simple case of adding new scenarios and updating the choreography to cater for these scenarios - this may cause dependency validation issues with service designs and implementations in the system being productised.

We could tag the new scenarios and choreography resource with a version - so when the validation is performed by the productisation team, these additional and changed resources are ignored. Or the productisation team could use a separate copy of the TIA project descriptor - but then what happens if they make changes, do these then just have to be merged back into the development streams version, as with other artifact changes?

I think my preference is that we treat the file as describing a particular version in time - and that version is declared as part of the (1) project level information. Each stream then works on its copy of the project file - but when appropriate, the development stream takes updates from the other stream(s). The only issue is what to do with signoff information, on resources that are changed as part of the subsequent work. Possibly the signoff information should be tied to the project version, but then each resource would also need to define which project version at which it was last changed.

I've read the previous thread on artifact dependency management and came away with one understanding. Reading through the fields proposed for the TIA project descriptor in this thread, I'm getting a different impression.

The main core purpose of the TIA project descriptor is as outlined in the other thread - to provide an explicit way to relate artifacts that are dependent, so that they can be validated as changes are made. The current implicit approach is not workable going forward, both in terms of handling the different types of artifacts, or the environments in which those artifacts may be stored (i.e. Eclipse, guvnor, etc).

Given that we are going to store this level of information within a document related to the project, it seemed sensible to see whether other uses could be made of the information (i.e. other than just using it for validation), and whether those other goals required further information to be stored in the document.

So one way to think about this descriptor is that it is a governing document for the full project lifecycle.

Thanks, Gary! That's the impression I got from the prior thread, but I got a bit confused when this thread added in the supporting fields for the other use cases. With that in mind, I'll toss in my $0.02.

A general issue will be how the information recorded in the tia is affected over revisions of the project and its artifacts. If review/signoff is recorded, then this needs to be tied to specific iterations. What information is explicitly recorded, and what information is subject to external version management.

I agree with your conclusion that each iteration should get its own tia file. This should simplify the internal structure of the tia, although it would allow multiple tias to exist in parallel. Part of retiring a version of a service (after a deprecation period, of course) would be to remove the old tia from source control.

At the risk of stating the obvious, if you put complete version history into the tia, you're going to have an increasingly large tia file over time. I can think of a couple of downsides to that.

Due to the varied locations in which this file may need to be used, it may be best to forget version management, other than as a way to track how the file contents changed over time. Therefore we need to consider how best to cater for project iterations and the impact it has on the information we record.

I agree with the part about externalizing version management. I may be reading too much into this, but it seems like you're talking about modeling interdependencies between iterations (e.g., iteration 1 makes some changes to the choreography based on a defect uncovered in integration testing, so how does that flow back into iteration 2 which is concurrently in development). If so, I'd suggest externalizing that process too for now.

We could tag the new scenarios and choreography resource with a version - so when the validation is performed by the productisation team, these additional and changed resources are ignored. Or the productisation team could use a separate copy of the TIA project descriptor - but then what happens if they make changes, do these then just have to be merged back into the development streams version, as with other artifact changes?

I think that it's reasonable to say that the SOA design & dev tool (i.e., Savara) will not try to merge changes forward from one iteration to the next. Supporting (with manual approval) that would be a great feature for a future version, but I think it would be hard to get the behavior right in the first version.

I think my preference is that we treat the file as describing a particular version in time - and that version is declared as part of the (1) project level information.

That makes sense to me.

Each stream then works on its copy of the project file - but when appropriate, the development stream takes updates from the other stream(s).

Yes, so long as that is a manual task and not something that the tool tries to figure out.

The only issue is what to do with signoff information, on resources that are changed as part of the subsequent work. Possibly the signoff information should be tied to the project version, but then each resource would also need to define which project version at which it was last changed.

Wouldn't sign-off information be monitored and enforced by the SOA registry/repository?

Due to the varied locations in which this file may need to be used, it may be best to forget version management, other than as a way to track how the file contents changed over time. Therefore we need to consider how best to cater for project iterations and the impact it has on the information we record.

I agree with the part about externalizing version management. I may be reading too much into this, but it seems like you're talking about modeling interdependencies between iterations (e.g., iteration 1 makes some changes to the choreography based on a defect uncovered in integration testing, so how does that flow back into iteration 2 which is concurrently in development). If so, I'd suggest externalizing that process too for now.

No, I don't think we need to consider dependencies between iterations, except possibly near to deployment when it may be useful from a change management perspective to understand the impact of changes that have occurred between two iterations of the project. But as you say, this should be an external process.

Brian Carothers wrote:

Each stream then works on its copy of the project file - but when appropriate, the development stream takes updates from the other stream(s).

Yes, so long as that is a manual task and not something that the tool tries to figure out.

Yes definitely. Just need some good comparison tools to help guide this manual task.

Brian Carothers wrote:

The only issue is what to do with signoff information, on resources that are changed as part of the subsequent work. Possibly the signoff information should be tied to the project version, but then each resource would also need to define which project version at which it was last changed.

Wouldn't sign-off information be monitored and enforced by the SOA registry/repository?

Good point - by having comparison tools that can perform an 'impact analysis' between iterations, it should be possible to provide a report that guides the sign-off decision, but the actual sign-off should occur in the repository.

I never imagined these being requirements of the Eclipse based tooling; but always thought of the management of relationships between project artifacts and their versioning and life cycle management and governance as a service repository feature, and that Savara would be dependent on a repository as part of the Overlord project. It seems to me like quite a bit of work to replicate what would be managed in the service repository. While JBoss does not currently have a service / SOA repository, the work I did with Guvnor (discussed in previous post), shows that Guvnor can be used for basic relationship management, versioning, and life-cycle management.

This is a good point. Introducing this descriptor would be an overhead, and therefore only worth doing if there was real need. Therefore I thought it would be a good idea to explore the pros and cons of relying on an SOA repository - which we can then use to dive deeper into each of the potential issues:

(1) Con: Means that no validation can be performed without firstly storing and associating the artifacts in the repository.

(3) Con: Assumes that the repository makes the dependency information easily available from within a development environment - unless this implies that the validation can only occur when the artifacts are stored back in the repository, which is not ideal. Although Guvnor may provide this metadata in the Eclipse workspace, it does not mean that all repositories will.

(4) Con: The value added information provided for the dependencies and project may not be stored in the repository(?). The project level info could be stored else where, but dependency info may be difficult - such as mapping information between the source and target artifacts to aid the comparison.

One solution to the overhead of duplicated information may be to support bi-direction dependency maintenance between the project descriptor and repository, so if links exist in one but not the other, then repair mechanism could resolve any issues. Similarly initial setup of links, either in the repo or project descriptor, could be driven by the other.

So for example, if we create some initial artifacts (choreography and associated scenarios), with the TIA project descriptor that defines the relationship between them in the Eclipse context, then when uploaded to the repository, the descriptor information can be used to automatically establish dependencies between the choreography and scenarios, and also update the descriptor with the resource locations in the repository.

I agree with Jeff that dependency management is traditionally the province of the service repository, but dependency management traditionally requires a significant amount of manual data entry that I'd rather avoid if we can.

If we store _limited_ dependency information in the TIA, then a service repository could be made "TIA-aware" and automatically update its dependencies when a service with a TIA file is uploaded. Service repositories that aren't TIA-aware could still have the dependencies from the TIA managed manually, as would be the case with any other dependency.

I strongly believe that the TIA information should be as limited as possible though for the first version. Reading some of the ideas earlier in the thread, the Savara plugin was starting to sound like Rational Rose c. 2001 - requirements management, design documentation, and code generation all in one place.

I agree with Gary's assessment above in general, and with point 3 in particular.