Bug 496299 - Controlled Units as Integral Fragments
Summary: Controlled Units as Integral Fragments
Status: VERIFIED FIXED
Alias: None
Product: Papyrus
Classification: Modeling
Component: Core (show other bugs)
Version: 2.0.0   Edit
Hardware: All All
: P2 enhancement (vote)
Target Milestone: 3.0.0   Edit
Assignee: Christian Damus CLA
QA Contact: Peter Cigehn CLA
URL:
Whiteboard:
Keywords:
Depends on: 497341 497342
Blocks: 497865
  Show dependency tree
 
Reported: 2016-06-17 08:59 EDT by Christian Damus CLA
Modified: 2017-05-10 04:19 EDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Damus CLA 2016-06-17 08:59:07 EDT
Up to and including the Neon release, Papyrus implements controlled units as "sub-models" which are intended to be openable and manipulatable independently in their own editors.  This implies that

* they do not have any kind of reference to their containing namespaces, otherwise some parent units would also be opened
* they redundantly apply all of the profiles that are applied on nesting packages, so that stereotype applications are still available independently

This facility can be convenient for certain scenarios, especially largely independent teamwork, but also can tend to lead to some kinds of maintenance challenges.  Most significant is the redundancy of profile applications, so that when dynamic profiles are used, sub-models can end up over time having a variety of different versions of the same profile applied and different even from the versions applied to the top or other nesting packages.  This is especially challenging in the case of references between stereotypes, because in a dynamic profile definition, the EClasses of different versions are distinct and incompatible, reference-wise.

Moreover, all of this sub-model capability really only works for Packages because:

* in the UML, only Packages are permitted not to have an owner
* only Packages can apply profiles

Papyrus does permit non-packages to be controlled (albeit with a warning), and so these are not actually independently openable and attempting to open them leads to problems.

Papyrus should offer an alternative controlled unit that is not intended to be independently openable and, when the user does attempt to open it, instead opens the root package or activates an editor that already has the root package open.  This kind of unit would be distinguished from the "sub-model" kind somehow by its content, so that this nature is available for indexing — the Papyrus model index would be extended with controlled-unit linkages — and an explicit or implicit reference to the parent unit chain so that all upward dependencies may be loaded before the sub-unit resource is loaded (an implicit reference would presumably rely on the index).  All non-package units that in Neon are of the "sub-model" variety would be transformed to this new style on demand; sub-models would remain as they are with a user action available to convert them.
Comment 1 Christian Damus CLA 2016-06-29 17:39:55 EDT
Prototype:  https://git.eclipse.org/r/76253
Demo video: https://youtu.be/EuWeTdy3zoY

The prototype so far implements a workspace-wide index of cross-resource containment references with awareness of resources that are controlled units behaving more like RSA's model fragments (I call the Papyrus analogue "model shards").  These shards are indicated by an annotation but, contrary to my remark in comment 0, there is no annotation reference or other mechanism that links a shard to its parent resource.  Instead, this prototype relies on the index, which has the complete controlled-unit topology in both directions.  This lets the editor open the root unit instead of the sub-unit and entirely avoids the problem of losing stereotype applications (bug 458837) when the parent resource is not loaded in time.  And, because this index also covers "normal" Papyrus controlled units, it can potentially be used to advantage in those use cases, too.
Comment 2 Peter Cigehn CLA 2016-06-30 08:28:31 EDT
My first gut feeling regarding using an index-based solution was: Does it scale? Knowing from experience any index-based solution will bump into scalability issues (even if find intelligent ways of ensuring that the index is updated in the background during startup and so on). What happens if you have an extremely large model(s) (which is probably the best scenario when shards are being useful), with a work-space with lots of projects with lots of models, where each such models has lots of shards, where it takes some considerable time to update the index. If you start navigating/loading the model before the index is up to date (or the index gets corrupted by some reason, e.g. instabilities in the surrounding environment), what happens then?

It just feels a bit more safe to base in on an explicit references to the parent shard/model, as well as the child shards, as it is done in legacy models.

Or did you look into this index-based solution since it was less "intrusive" in the current design/architecture?
Comment 3 Peter Cigehn CLA 2016-06-30 09:34:19 EDT
I am unable to access the Gerrit draft. Not sure what access rights are needed, or if it is only the author that can access Gerrit drafts.
Comment 4 Christian Damus CLA 2016-06-30 12:20:22 EDT
(In reply to Peter Cigehn from comment #2)
> My first gut feeling regarding using an index-based solution was: Does it
> scale? Knowing from experience any index-based solution will bump into
> scalability issues (even if find intelligent ways of ensuring that the index
> is updated in the background during startup and so on). What happens if you

Well, I would like to say that the index already seems to scale well enough because it is used for the external stereotype applications.  :-)  But, I haven't actually had the experience of testing it on a workspace quite as large as yours, I think.

It is still in my plans to add persistence of the index between sessions for quick initialization.  Perhaps this use case will motivate that additional enhancement.


> have an extremely large model(s) (which is probably the best scenario when
> shards are being useful), with a work-space with lots of projects with lots
> of models, where each such models has lots of shards, where it takes some
> considerable time to update the index. If you start navigating/loading the
> model before the index is up to date (or the index gets corrupted by some
> reason, e.g. instabilities in the surrounding environment), what happens
> then?

The index is initialized in the background from start-up of the workbench.  After that, it is updated incrementally as resources are saved.  This much is quite efficient, because the index data are partitioned per resource URI, so the impact is localized to the modified resources only.

Any user action that absolutely requires the index will be blocked until any updates pending in it are complete.  Of interest to us here is that one of those user actions now is the opening of a model editor, as it needs to know what is the root resource to open, and that cannot be determined until all resources are indexed.


> It just feels a bit more safe to base in on an explicit references to the
> parent shard/model, as well as the child shards, as it is done in legacy
> models.

Well, this index is also useful now for the other mode of controlled unit in Papyrus, which must not have such linkage annotations because those would artificially add proxies which resolution would open the parent resource chain.

Also, these annotation references wouldn't be sufficient anyways to avoid the kinds of problems that are represented by bug 458837.  The editor needs to know what is the root unit (containing profile applications) to load a priori; it is not enough to let it be loaded by resolution of references from the sub-unit because that is too late to rescue stereotype applications that were already discarded.


> Or did you look into this index-based solution since it was less "intrusive"
> in the current design/architecture?

Not so much, no.  Mostly because it was a component that provides benefits to other use cases besides and because it provides a solution to the problem of the dependency between stereotype applications and profile applications.
Comment 5 Christian Damus CLA 2016-06-30 12:22:20 EDT
(In reply to Peter Cigehn from comment #3)
> I am unable to access the Gerrit draft. Not sure what access rights are
> needed, or if it is only the author that can access Gerrit drafts.

That's annoying.  The point of a draft is that others can have a look at it.

I've added you as a reviewer.  I have no confidence that this will work.  If it doesn't, I'll raise a bugzilla for webmaster.
Comment 6 Peter Cigehn CLA 2016-07-01 03:54:54 EDT
(In reply to Christian W. Damus from comment #5)
> (In reply to Peter Cigehn from comment #3)
> > I am unable to access the Gerrit draft. Not sure what access rights are
> > needed, or if it is only the author that can access Gerrit drafts.
> 
> That's annoying.  The point of a draft is that others can have a look at it.
> 
> I've added you as a reviewer.  I have no confidence that this will work.  If
> it doesn't, I'll raise a bugzilla for webmaster.

After being added as a reviewer I was able to access the Gerrit change via the web-interface, since I am logged in via the web-interface. But when trying to fetch the Gerrit change via EGit into my development environment it still failed. I guess that the way things are setup in my environment, it does not seem to provide my credentials during the fetch (either an issue with my setup or possibly with EGit). Anyway, it seem to partly work when being added as a reviewer.
Comment 7 Peter Cigehn CLA 2016-07-01 04:17:51 EDT
(In reply to Christian W. Damus from comment #4)
> (In reply to Peter Cigehn from comment #2)
> > My first gut feeling regarding using an index-based solution was: Does it
> > scale? Knowing from experience any index-based solution will bump into
> > scalability issues (even if find intelligent ways of ensuring that the index
> > is updated in the background during startup and so on). What happens if you
> 
> Well, I would like to say that the index already seems to scale well enough
> because it is used for the external stereotype applications.  :-)  But, I
> haven't actually had the experience of testing it on a workspace quite as
> large as yours, I think.
> 
> It is still in my plans to add persistence of the index between sessions for
> quick initialization.  Perhaps this use case will motivate that additional
> enhancement.
> 

Okay, fair enough. Since we have other features relying on the index, maybe the scalability issue needs to be handled/improved anyway as you say. I just get a bad feeling when thinking about really large workspaces with potentially hundreds of projects, potentially hundreds of models, with potentially thousands of shards (or even more). But in those scenarios, knowing from experience, we probably have a lot of other scalability issues to consider anyway... :)
Comment 8 Peter Cigehn CLA 2016-07-01 08:06:50 EDT
I just had a brief chat with Patrik, and one thing that is important is that sharded models must be possible to open in head-less environment, and even in non-OSGi environments, i.e. plain Java, e.g. in code-generators and other transformation tools. Can we in those situations even rely on a work-space based index?

I add Ernesto to get his view on how the command-line code-generator for Papyrus-RT relies on any Papyrus framework for opening the model during code-generation.

If we consider the scenarios described in Bug 458837, there is need to be able to open those kinds of models, e.g. as described in Bug 458837 Comment 6.
Comment 9 Ernesto Posse CLA 2016-07-01 10:48:44 EDT
(In reply to Peter Cigehn from comment #8)
> I just had a brief chat with Patrik, and one thing that is important is that
> sharded models must be possible to open in head-less environment, and even
> in non-OSGi environments, i.e. plain Java, e.g. in code-generators and other
> transformation tools. Can we in those situations even rely on a work-space
> based index?
> 
> I add Ernesto to get his view on how the command-line code-generator for
> Papyrus-RT relies on any Papyrus framework for opening the model during
> code-generation.
> 
> If we consider the scenarios described in Bug 458837, there is need to be
> able to open those kinds of models, e.g. as described in Bug 458837 Comment
> 6.

The standalone generator opens the Papyrus model just as any Ecore file since the .uml is an Ecore model. We only need to make the necessary UML resource initialization, registration of the UML-RT profile, Ansi C library, RtCppProperties profile and RTS Model Library and set the appropriate URI mappings.

We don't do anything special regarding models imported from RSA, and I have not tried it so I don't know if it would work. In particular, we don't trigger a migration operation, or any kind of Stereotype repair as described in Bug 458837 Comment 6.

If this is something that should be supported, perhaps there should be an explicit bug targeting the standalone code generator. 

Keep in mind that since the standalone does not run Eclipse, we cannot use many of the Eclipse functionality that requires the Eclipse runtime running, such as the workspace-based index, or whatever is used by the migration plugin. Basically we can rely only on what EMF provides, e.g. URI-based references. In particular, if the migration uses the API in org.eclipse.papyrusrt.umlrt.core, then we won't be able to do anything with that migration, since, as discussed in the mailing list a couple of weeks ago (under "Reusing core components"), the classes in the "core" component use "ElementTypeUtil" which requires Papyrus running, and therefore cannot be used in standalone.
Comment 10 Peter Cigehn CLA 2016-07-01 11:14:10 EDT
(In reply to Ernesto Posse from comment #9)
> The standalone generator opens the Papyrus model just as any Ecore file
> since the .uml is an Ecore model. We only need to make the necessary UML
> resource initialization, registration of the UML-RT profile, Ansi C library,
> RtCppProperties profile and RTS Model Library and set the appropriate URI
> mappings.
> 
> We don't do anything special regarding models imported from RSA, and I have
> not tried it so I don't know if it would work. In particular, we don't
> trigger a migration operation, or any kind of Stereotype repair as described
> in Bug 458837 Comment 6.

Just to be clear: Any code-generator (whether it is the Papyrus-RT one or any own developed transforms) should not have to treat imported models in any special way, that is the responsibility of the import tool to ensure that models are ready to be used for any Papyrus-based tooling.

Sorry for bringing you into the discussion without any prior explanation about the core issue here (and it is a non-trivial aspect, so I am not sure that I can explain).

All I wanted to know was in what way the code-generator opens models and what core services/frameworks from Papyrus the code-generator is using. The core essence of my question is of course that the code-generator in general should not have to bother about any shared models that we discuss here.

> 
> If this is something that should be supported, perhaps there should be an
> explicit bug targeting the standalone code generator. 
> 
> Keep in mind that since the standalone does not run Eclipse, we cannot use
> many of the Eclipse functionality that requires the Eclipse runtime running,
> such as the workspace-based index, or whatever is used by the migration
> plugin. Basically we can rely only on what EMF provides, e.g. URI-based
> references. In particular, if the migration uses the API in
> org.eclipse.papyrusrt.umlrt.core, then we won't be able to do anything with
> that migration, since, as discussed in the mailing list a couple of weeks
> ago (under "Reusing core components"), the classes in the "core" component
> use "ElementTypeUtil" which requires Papyrus running, and therefore cannot
> be used in standalone.


Just to be clear again: The code-generator should not have to bother about any migration aspects. All that should already have been handled by the migration tool. Keep in mind also that the sharded models discussed here very well could be created directly in Papyrus-RT (and thus it does not necessarily have anything to do with migrated models).
Comment 11 Patrik Nandorf CLA 2016-07-04 04:21:28 EDT
(In reply to Ernesto Posse from comment #9)
> (In reply to Peter Cigehn from comment #8)
> Keep in mind that since the standalone does not run Eclipse, we cannot use
> many of the Eclipse functionality that requires the Eclipse runtime running,
> such as the workspace-based index, 

Ok, how do you load and generate from RT models with an externalized RT profile?

> Basically we can rely only on what EMF provides, e.g. URI-based
> references. In particular, if the migration uses the API in
> org.eclipse.papyrusrt.umlrt.core, then we won't be able to do anything with
> that migration, since, as discussed in the mailing list a couple of weeks
> ago (under "Reusing core components"), the classes in the "core" component
> use "ElementTypeUtil" which requires Papyrus running, and therefore cannot
> be used in standalone.

Ok, I haven't looked into the org.eclipse.papyrusrt.umlrt.core but I assume this contains contains classes and methods to programmatically read and manipulate and 
RT elements (such as capsules and prototcols)? Not having this outside OSGi would mean that we have to re-implement this on out own with isn't a viable solution.

How do you you handle reading of RT elements in your standalone code generator? Are there any helper methods available?
Comment 12 Patrik Nandorf CLA 2016-07-04 04:30:53 EDT
(In reply to Peter Cigehn from comment #10)
> (In reply to Ernesto Posse from comment #9)
> 
> All I wanted to know was in what way the code-generator opens models and
> what core services/frameworks from Papyrus the code-generator is using. The
> core essence of my question is of course that the code-generator in general
> should not have to bother about any shared models that we discuss here.
> 

To add to this, I see two aspects, (1) model loading and (2) methods for reading and possibly manipulating the loaded model

1. loading the model need the support for 'fragments' so a possible fragmented model is loaded into one 'logical' model.

2. when the model is loaded (possibly lazily) there needs to be mechanisms ("helpers", possibly in org.eclipse.papyrusrt.umlrt.core) to read and manipulate the model, for use e.g .in code generators, model validation etc.

Both of these needs to be supported also outside OSGi
Comment 13 Christian Damus CLA 2016-07-04 07:35:16 EDT
This prototype does not interfere in any way with the usual cross-resource containment proxies of EMF and their resolution to load "shard" resources.  Loading models from the top down will work correctly in any environment supported by EMF.

I think in any case one would want to require transformations and other off-line/Eclipse-free processes to load models from the root resource.

It is only for editing scenarios that we want (I think) to provide the convenience of opening sub-model units in their whole model context.  In any other case, I think it's quite appropriate to let the user be the "index" and require operations to be invoked on the root unit.

And to support this editing convenience by relying on upward linkages in annotations, we would need to have a pre-loading step in which the ModelSet (or client application) first detects that there is a parent resource to be loaded (and recursively up the chain) to ensure that the model is loaded from the top down, *before* any normal EMF-ish loading of resources is performed.  One of the problems that we have is that by just loading a sub-model unit into the ModelSet, its stereotype applications are already broken and become detached from their base elements.  The index just obviates this on-demand pre-processing.
Comment 14 Peter Cigehn CLA 2016-07-04 07:51:49 EDT
I am not sure that I follow all the details here, but I guess that even in the scenario where you always start loading a model specific model from the root, e.g. in the headless codegeneration/transformation scenario, I guess you will bump into issues where you also need to traverse upwards to the root of another model.

Take the following example with a sharded model: S with shards S1 and S2, i.e. S -> S1 -> S2.

If you then have another (also potentially sharded) model with a reference to a model element contained in S2, i.e. M -> S2, then you need to follow the upwards S2 -> S1 -> S to be able to ensure the correct stereotype applications of S2 (and S1) based on the profile application in S.

Or do mean that this scenario works as expected? I guess it is exactly this kind of scenario that causes the issues in Bug 458837 Comment 7 (with an example model attached similar to what I describe above).
Comment 15 Christian Damus CLA 2016-07-04 08:40:31 EDT
(In reply to Peter Cigehn from comment #14)

There are two different cases here:

(a) a sub-model unit ("shard") contained in two different models (multiple root units)

(b) a cross-reference from a resource in one model to a resource in another model

In the first case, the index in the prototype currently handles that, but not the editor.  The editor just needs to ensure that all root models are loaded.  I'm not sure yet what the impact on read-only handling would be:  should all models but the first be treated as read-only (except for elements in resources that they share) or should all models be writable by default (because they are, properly speaking, all one logical model)?  I'm inclined to the latter interpretation.

In the second case, we would like some kind of intercept of the proxy resolution that loads the referenced model from the top down in the same way as the editor originally did for the first model.


> I am not sure that I follow all the details here, but I guess that even in
> the scenario where you always start loading a model specific model from the
> root, e.g. in the headless codegeneration/transformation scenario, I guess
> you will bump into issues where you also need to traverse upwards to the
> root of another model.

If the transformation needs all of these models in order to do its work, it should be given them all to load a priori.


> Take the following example with a sharded model: S with shards S1 and S2,
> i.e. S -> S1 -> S2.
> 
> If you then have another (also potentially sharded) model with a reference
> to a model element contained in S2, i.e. M -> S2, then you need to follow
> the upwards S2 -> S1 -> S to be able to ensure the correct stereotype
> applications of S2 (and S1) based on the profile application in S.

That's right.


> Or do mean that this scenario works as expected? I guess it is exactly this
> kind of scenario that causes the issues in Bug 458837 Comment 7 (with an
> example model attached similar to what I describe above).

No, my prototype hadn't got this far, yet.  That's case (b) that I described above.

If we want a solution that works without the index, we're going to have to fiddle with resource loading in the Papyrus ModelSet, and as far as I know the ModelSet, itself, requires the Eclipse run-time, so that doesn't work for headless applications (why are they constrained to run without the Eclipse run-time?  Eclipse platform supports headless applications, too).  So, the specialized resource loading based on some kind of parent-resource reference could be implemented in some more general kind of ResourceSet that is suitable for use outside of the Eclipse context.
Comment 16 Peter Cigehn CLA 2016-07-04 09:01:37 EDT
(In reply to Christian W. Damus from comment #15)
> (In reply to Peter Cigehn from comment #14)
> 
> There are two different cases here:
> 
> (a) a sub-model unit ("shard") contained in two different models (multiple
> root units)
> 
> (b) a cross-reference from a resource in one model to a resource in another
> model
> 
> In the first case, the index in the prototype currently handles that, but
> not the editor.  The editor just needs to ensure that all root models are
> loaded.  I'm not sure yet what the impact on read-only handling would be: 
> should all models but the first be treated as read-only (except for elements
> in resources that they share) or should all models be writable by default
> (because they are, properly speaking, all one logical model)?  I'm inclined
> to the latter interpretation.
> 

But if we compare this with the legacy fragment concept, then case (a) can never occur. A fragment in the legacy tooling is contained in one, and only one, model at any time. As we have discussed this also ensures the name space semantic of that sub-model (or "shard").

Yes, the current Papyrus sub-model concept, can potentially be contained in two different models (but the UI does not (yet?) support this, since you can never "import" a sub-model into another model, only create a new sub-model from an existing model, in practice making a sub-model always to be contained in one and only one model).

But should we even support case (a)? Especially if we are keen on keeping the name space semantic? If you really want to have a sub-model in two different models, then you probably should use element/package import (depending on what name space semantic you want).

Sure, if we still use an index-based solution, we can never ensure that case (a) never occurs (for the legacy fragment you only have one, and only one, parent EAnnotation reference so you can ensure that a fragment is only contained in one and only one parent model), so we need to have some reasonable behavior as you say whenever that happens.
Comment 17 Christian Damus CLA 2016-07-04 09:12:44 EDT
(In reply to Peter Cigehn from comment #16)
> (In reply to Christian W. Damus from comment #15)
> > (In reply to Peter Cigehn from comment #14)
> > 
> > There are two different cases here:
> > 
> > (a) a sub-model unit ("shard") contained in two different models (multiple
> > root units)
> > 
> > (b) a cross-reference from a resource in one model to a resource in another
> > model
> > 
> > In the first case, the index in the prototype currently handles that, but
> > not the editor.  The editor just needs to ensure that all root models are
> > loaded.  I'm not sure yet what the impact on read-only handling would be: 
> > should all models but the first be treated as read-only (except for elements
> > in resources that they share) or should all models be writable by default
> > (because they are, properly speaking, all one logical model)?  I'm inclined
> > to the latter interpretation.
> > 
> 
> But if we compare this with the legacy fragment concept, then case (a) can
> never occur. A fragment in the legacy tooling is contained in one, and only
> one, model at any time. As we have discussed this also ensures the name
> space semantic of that sub-model (or "shard").

Indeed, I just wanted to be sure that we aren't conflating two different kinds of structure in our discussion.  I have no interest in supporting case (a); I hope nobody asks for it!  I feel that the namespace relationship is a very strong dependency, and namespaces are unique (this is, in fact, explicit in the UML metamodel: Fig 7.5 Namespaces, chapter 7.4.2).

And in any case, EMF would balk when two resources are loaded in a ResourceSet that both have containment proxies resolving to the same object:  EMF implements single containment, so the object would be removed from its first container as soon as the second containment proxy is resolved.
Comment 18 Peter Cigehn CLA 2016-07-04 09:30:36 EDT
(In reply to Christian W. Damus from comment #17)
> (In reply to Peter Cigehn from comment #16)
> > (In reply to Christian W. Damus from comment #15)
> > > (In reply to Peter Cigehn from comment #14)
> > > 
> > > There are two different cases here:
> > > 
> > > (a) a sub-model unit ("shard") contained in two different models (multiple
> > > root units)
> > > 
> > > (b) a cross-reference from a resource in one model to a resource in another
> > > model
> > > 
> > > In the first case, the index in the prototype currently handles that, but
> > > not the editor.  The editor just needs to ensure that all root models are
> > > loaded.  I'm not sure yet what the impact on read-only handling would be: 
> > > should all models but the first be treated as read-only (except for elements
> > > in resources that they share) or should all models be writable by default
> > > (because they are, properly speaking, all one logical model)?  I'm inclined
> > > to the latter interpretation.
> > > 
> > 
> > But if we compare this with the legacy fragment concept, then case (a) can
> > never occur. A fragment in the legacy tooling is contained in one, and only
> > one, model at any time. As we have discussed this also ensures the name
> > space semantic of that sub-model (or "shard").
> 
> Indeed, I just wanted to be sure that we aren't conflating two different
> kinds of structure in our discussion.  I have no interest in supporting case
> (a); I hope nobody asks for it!  I feel that the namespace relationship is a
> very strong dependency, and namespaces are unique (this is, in fact,
> explicit in the UML metamodel: Fig 7.5 Namespaces, chapter 7.4.2).
> 
> And in any case, EMF would balk when two resources are loaded in a
> ResourceSet that both have containment proxies resolving to the same object:
> EMF implements single containment, so the object would be removed from its
> first container as soon as the second containment proxy is resolved.

As we discussed off-line, comparing with how controlled units were used in RoseRT, I could see a potential use of (a) when structuring and re-using subsets of large models, to build up smaller subset models.

But sure, as you say, the user must ensure that those models which bring in the same shard, never is loaded into the same resource set, since you still have single containment in EMF. And those models that bring in the same shard, probably should bring it into the same name space, i.e. those models needs to mirror each other top level name spaces, to keep the name space semantics.
Comment 19 Ernesto Posse CLA 2016-07-04 10:02:02 EDT
(In reply to Patrik Nandorf from comment #11)
> (In reply to Ernesto Posse from comment #9)
> > (In reply to Peter Cigehn from comment #8)
> > Keep in mind that since the standalone does not run Eclipse, we cannot use
> > many of the Eclipse functionality that requires the Eclipse runtime running,
> > such as the workspace-based index, 
> 
> Ok, how do you load and generate from RT models with an externalized RT
> profile?

I'm not sure what you mean by an "externalized RT profile". The input .uml file is just an Ecore file which includes all the profile applications and stereotypes. All I need to do is to invoke the proper UML resource initialization, loading and registration of the RT profile and libraries, and then I load the model. The loading of the profiles, libraries and model are all the same: basic EMF resource loading. Any references in the model to a profile are proxies and since the profiles are registered during initialization, EMF resolves them.

FYI the source code is in the org.eclipse.papyrusrt.codegen.standalone plugin, and the main class is 'StandaloneUMLRTCodeGenerator'. In particular, you can check the 'init' method.

During generation, we have an API to represent elements from the UML-RT profile. This API includes methods such as 'isCapsule', which check the stereotype of the EObject. As I said above, references to these stereotypes are proxies which are resolved by EMF, given that the profile registration has been made.

> > Basically we can rely only on what EMF provides, e.g. URI-based
> > references. In particular, if the migration uses the API in
> > org.eclipse.papyrusrt.umlrt.core, then we won't be able to do anything with
> > that migration, since, as discussed in the mailing list a couple of weeks
> > ago (under "Reusing core components"), the classes in the "core" component
> > use "ElementTypeUtil" which requires Papyrus running, and therefore cannot
> > be used in standalone.
> 
> Ok, I haven't looked into the org.eclipse.papyrusrt.umlrt.core but I assume
> this contains contains classes and methods to programmatically read and
> manipulate and 
> RT elements (such as capsules and prototcols)? Not having this outside OSGi
> would mean that we have to re-implement this on out own with isn't a viable
> solution.

Exactly. The 'core' plugin has such classes and methods, but as I discovered, they rely on an 'Element Type' registry which is handled by Papyrus, and it requires Papyrus running. For this reason, the code generator uses a different API, which does not rely on such registry. The downside is that there is some code duplication between the code generator and tooling. The e-mail thread I referred to was precisely on how to avoid such duplication, but the conclusion was that as long as Papyrus-RT tooling uses the 'Element Type'-based API, we have no choice but to have a duplicated API.

> How do you you handle reading of RT elements in your standalone code
> generator? Are there any helper methods available?

Yes, the API I'm referring to is in 'org.eclipse.papyrusrt.xtumlrt.util', in two classes: UMLRealTimeProfileUtil and UMLRealTimeStateMachProfileUtil. They are essentially a facade of the profiles.

To give an example, we have this:

	public static boolean isCapsule(Element el) {
		Stereotype s = el.getApplicableStereotype(UML_REAL_TIME_CAPSULE);
		return s != null && el.isStereotypeApplied(s);
	}

where Element is an EObject of type org.eclipse.uml2.uml.Element, in the model, and UML_REAL_TIME_CAPSULE is the String "UMLRealTime::Capsule".
Comment 20 Peter Cigehn CLA 2016-07-04 10:16:29 EDT
(In reply to Ernesto Posse from comment #19)
> I'm not sure what you mean by an "externalized RT profile". 

A good place to check this is:

http://help.eclipse.org/neon/index.jsp?topic=%2Forg.eclipse.papyrus.uml.decoratormodel.doc%2Ftarget%2Fgenerated-eclipse-help%2Fprofileapplications.html&cp=61_0_1_6_7

One good example of a profile application where you could want to store it in a separate resource is the RtCppProperties profile, e.g. in a scenario where you have different sets of stereotype applications depending on which target/deployment scenario you are generating code for.

So the Papyrus-RT code generator must be able to handle externalized profile applications. I guess one should be able to specify which externalized profile application to be used for a specific build. In the legacy tooling this can be done using the transformation configuration file that controls the code-generator.
Comment 21 Ernesto Posse CLA 2016-07-04 10:24:13 EDT
(In reply to Peter Cigehn from comment #20)
> (In reply to Ernesto Posse from comment #19)
> > I'm not sure what you mean by an "externalized RT profile". 
> 
> A good place to check this is:
> 
> http://help.eclipse.org/neon/index.jsp?topic=%2Forg.eclipse.papyrus.uml.
> decoratormodel.doc%2Ftarget%2Fgenerated-eclipse-help%2Fprofileapplications.
> html&cp=61_0_1_6_7
> 
> One good example of a profile application where you could want to store it
> in a separate resource is the RtCppProperties profile, e.g. in a scenario
> where you have different sets of stereotype applications depending on which
> target/deployment scenario you are generating code for.
> 
> So the Papyrus-RT code generator must be able to handle externalized profile
> applications. I guess one should be able to specify which externalized
> profile application to be used for a specific build. In the legacy tooling
> this can be done using the transformation configuration file that controls
> the code-generator.

Oh, I see, the profile application(s) and stereotypes are actually stored separately in that case?

I don't think the code generator supports that, unless the UML2 API itself supports this. I don't know how Papyrus represents such externalized profile applications. But if they work as standard EMF cross-references between different resources, it might work already "out-of-the-box". In any case I need to check this. Could you open a bug on the code generator for this?

Thanks.
Comment 22 Peter Cigehn CLA 2016-07-04 10:53:50 EDT
(In reply to Ernesto Posse from comment #21)
> I don't think the code generator supports that, unless the UML2 API itself
> supports this. I don't know how Papyrus represents such externalized profile
> applications. But if they work as standard EMF cross-references between
> different resources, it might work already "out-of-the-box". In any case I
> need to check this. Could you open a bug on the code generator for this?

I wrote Bug 497243 to track this.
Comment 23 Eclipse Genie CLA 2016-07-13 16:11:49 EDT
Gerrit change https://git.eclipse.org/r/76253 was merged to [streams/2.0-maintenance].
Commit: http://git.eclipse.org/c/papyrus/org.eclipse.papyrus.git/commit/?id=26fa306eb52451bee9529120c650f99b50a3ee92
Comment 24 Eclipse Genie CLA 2016-07-13 17:49:55 EDT
New Gerrit change created: https://git.eclipse.org/r/77281
Comment 26 Christian Damus CLA 2016-07-13 18:36:18 EDT
(In reply to Eclipse Genie from comment #23)
> Gerrit change https://git.eclipse.org/r/76253 was merged to
> [streams/2.0-maintenance].

(In reply to Eclipse Genie from comment #25)
> Gerrit change https://git.eclipse.org/r/77281 was merged to [master].
Comment 27 Peter Cigehn CLA 2016-07-14 06:59:31 EDT
Verified in the context of the latest Papyrus-RT build, based on the latest Papyrus Neon build, that the shard concept works as expected when testing on an importing a legacy UML-RT model with lots of fragments.

When opening such a shard/fragment, it opens in the context of its root model where the profile application is made. Opening a second shard/fragment part of the same root model, reused the already open model editor.

UI operations for manually splitting a model into shards/fragments or absorbing shards/fragments I assume is tracked by Bug 497865.
Comment 28 Eclipse Genie CLA 2016-07-15 08:47:08 EDT
New Gerrit change created: https://git.eclipse.org/r/77389
Comment 29 Eclipse Genie CLA 2016-07-15 08:47:33 EDT
New Gerrit change created: https://git.eclipse.org/r/77390
Comment 31 Eclipse Genie CLA 2016-07-15 09:20:21 EDT
Gerrit change https://git.eclipse.org/r/77389 was merged to [streams/2.0-maintenance].
Commit: http://git.eclipse.org/c/papyrus/org.eclipse.papyrus.git/commit/?id=9befdd5ea0fbc7a5a3b531578ab5118d709bfc7d
Comment 32 Eclipse Genie CLA 2016-07-19 11:44:52 EDT
New Gerrit change created: https://git.eclipse.org/r/77539
Comment 33 Eclipse Genie CLA 2016-07-19 12:08:38 EDT
Gerrit change https://git.eclipse.org/r/77539 was merged to [streams/2.0-maintenance].
Commit: http://git.eclipse.org/c/papyrus/org.eclipse.papyrus.git/commit/?id=930ebcac8c74bc6d08b8ff42c906bac252a2b0c3
Comment 34 Eclipse Genie CLA 2016-07-19 12:08:50 EDT
New Gerrit change created: https://git.eclipse.org/r/77541